Analyzing or emulating a keyboard performance using audiovisual dynamic point referencing

ABSTRACT

The systems and methods taught herein are generally directed to a dynamic point referencing of an audiovisual performance for an accurate and precise selection and controlled cycling of portions of the performance. The dynamic point referencing can be used by a learning artist, for example, in analyzing or performing a portion of the work through an accurate and precise digital audio/video instructional method having the controlled cycling feature. Such systems and methods will be appreciated, for example, by musicians, dancers, and other enthusiasts of the performing arts.

CROSS REFERENCE TO RELATED APPLICATIONS

This application is a continuation of U.S. application Ser. No.15/673,910, filed Aug. 10, 2017, which is a continuation of U.S.application Ser. No. 14/465,032, filed Aug. 21, 2014, now U.S. Pat. No.9,761,151, which is a continuation of U.S. application Ser. No.13/274,293, filed Oct. 14, 2011, now U.S. Pat. No. 8,847,053, whichclaims the benefit of U.S. Provisional Patent Application Ser. No.61/455,176, filed Oct. 15, 2010, each of which is hereby incorporatedherein by reference in its entirety.

BACKGROUND Field of the Invention

The systems and methods taught herein are generally directed to adynamic point referencing of an audiovisual performance for an accurateand precise selection and controlled cycling of portions of theperformance.

Description of the Related Art

Any performing artist can appreciate rhythm and timing, as well as thefact that many skills can be handed down from one artist to the next,whether the artists are aspiring or seasoned. Such performing artistscan include, for example, musicians, actors, dancers, and the like, andwill appreciate the teachings set-forth herein. Even the smallestinflections of a note's volume, attack, and length, for example, canrepresent some of the many reasons why one player could be consideredmore accomplished than another.

Performing artists currently use instructional DVD's or streaming videoto hone their skillset, learn new material, or acquire new skills.However, the problem remains that conventional video playback devicesand internet streaming players lack the ability of a transport systemthat can give a user the type accuracy, precision, and control toquickly identify and repeat sections that are difficult to learn.Traditional video playback systems simply do not provide a method toaccurately and precisely control and cycle sections of a video playbackin a rhythmic and desirable manner. The technology to do so is currentlyunavailable, for at least the reason that the performing arts tend tohave at least minor inconsistencies, as well as often complex andvariable timing, associated with a performance, making it difficult toautomate the parsing of the very unpredictable and highly detailedstructure of the performance. It should be noted that even the slightestinconsistencies or variations in the timing of a performance make itvery difficult, and in some case seemingly impossible, to automate theparsing of the performance.

A problem in the art includes learning how to emulate subtleties, suchas gestures, body movements, and hand positions, and the like, and thenidentifying the desired portion of a performance having the subtlety, aswell as accurately and precisely selecting and practicing the desiredportion until it is learned. The guitar, for example, as well as otherstringed instruments, is prone to subtle variations of the vibrato onthe strings, which can make performances unique from one player to thenext. Any musical instrument, or dance, can be subject to suchsubtleties where dance, of course, can be seen as directed to theaesthetics of combinations of body movements that follow a piece ofmusic. Drummers, percussionists, and other types of rhythm instrumentsalso include subtleties, for example, which may not be apparent to thelayman but can again be a determining factor as to why one musician maysound more accomplished. One of skill in any performing art canappreciate the fact that such subtleties can occur quickly, and a toolthat helps them identify and view the subtlety would be very helpful indeveloping or honing a skill.

In a desirable learning environment, a performer could easily, as wellas accurately and precisely, select a desired portion of an audio/videofile, repeatedly cycle the desired portion, adjust the tempo of thedesired portion according to the performer's skill level to make thelearning process easier, select one or more additional desired portionsas needed to learn the piece, and observing each desired portion at theadjusted tempo. Musicians try to achieve this type of system, makeshift,using the only tools they have available currently. For example, amusician may try to learn a section of a musical piece with videoassistance by trying to set a cycle point on their DVD player, mediaplayer, video editing software or video stream, but this usually resultsin many failed, and highly frustrating, attempts at accurately selectinga particular section of the work that they are learning. There is nosystem available, even a professional editing system, that provides atransport mechanism that can quickly, accurately, and precisely select aparticular, desirable rhythmical portion of an audiovisual performance,whether the performance varies by a minor amount or dramatically intempo throughout the performance. As such, it's apparent that thelearning artist is currently highly frustrated in his attempts atestablishing a repetitive section that repeats in a rhythmic pattern. Toadd to the frustration, the user has to deal with this problemrepeatedly while moving through a work, selecting each desired portionas he moves his way through the process of learning how to perform theentire work.

The above features can be somewhat achieved in a live environment, wherethe learning is done real-time with a live teacher, although theaccuracy and precision of the playback of the desired portion, the tempoof the playback, and the cycling is limited to human error and tolerancein the live environment. Unfortunately, not all musicians have readilyavailable access to the live teaching environment and, moreover, thosethat may have such access will not likely have the flexibility orfrequency of access that they may want. Moreover, access to instructionregarding a particular performance by a popular artist may be entirelyunavailable.

Visual displays are an effective way for a performing artist to learn,and this is particularly true for performing artists that cannot readmusic, for example, or don't have access to a teacher. Visualdemonstrations can provide a performing artist with general techniques,such as how to hold an instrument, hand positions, and body positions,as well as detailed information about fingering, nuances, and othertechniques used by performers. Moreover, subjective details in musicalnotation, for example, can also be observed while watching anaccomplished performing artist.

Currently, no system offers these features while also providing thefunction of observing a desired portion of a performance whilerhythmically cycled at an adjusted tempo. Generally speaking, those ofskill in the art of teaching movements that relate to a performing artwould appreciate a system that allows a user to select and view aperformer's actions according to a desired portion of performance,particularly where the desired portion contains difficult passageshaving many actions in rapid succession, perhaps moving too quickly forthe user to otherwise comprehend. As such, the system could findapplication in teaching any performing art such as, for example,musicians or dancers.

Accordingly, one of skill will appreciate having a learning tool thatprovides a unique and effective way to, at least, (i) learn intricateinterpretations of a performance; (ii) learn skills that can be variablefrom performer to performer; (iii) make an accurate and preciseselection of repetitive cycling points of a desired audiovisual portionof a performance; (iv) help those that cannot read music to identify,isolate, set rhythmical cycling points, and practice portions of amusical work they are trying to learn, whether these portions are foundin individual notes, beats, bars, or otherwise; (v) quickly referenceand mark different sections of a work; (vi) select repetitive cyclepoints that snap to a tempo/index grid, and easily adjustable toadditional cycling points; (vii) slow the audiovisual display down to acomfortable speed without changing the pitch; and, (viii) use the stepadvance mode to move between pre-selected beats or subdivided beats.

SUMMARY

The systems and methods taught herein are generally directed to adynamic point referencing of an audiovisual performance for an accurateand precise selection and controlled cycling of portions of theperformance. The dynamic point referencing can be used by a user inanalyzing or performing a portion of the work through an accurate andprecise digital audio/video instructional method having the controlledcycling feature. Such systems and methods will be appreciated, forexample, by musicians, dancers, and other enthusiasts of the performingarts.

In some embodiments, the teachings are directed to an instructionalaudiovisual work for teaching a user how to perform at least a portionof the work through an accurate and precise audiovisual method. In theseembodiments, the work comprises a video portion and an audio portion,wherein the work includes a performance by a performing artist. The workcan also comprise a dynamic point reference within the work for anaccurate and precise selection and point-cycling of a desired portion ofthe work by the user. In some embodiments, the dynamic point referenceincludes a tempo map of the audio portion that is adaptable for aplurality of tempos within the work.

The performer can be an amateur or a professional. For example, theperformer can be a popular artist, in some embodiments. Likewise, thework can be an amateur work or a professional work. And, the work can bean original, popular musical work in some embodiments.

An advantage of the teachings provided herein is the accuracy andprecision available in the selection of a desired portion. Due to thisadvantage, the cycling can include any selected time-frame within thework, such as a period of silence in an audio track, or any one or anyseries of sounds, beats, and bars. In some embodiments, the cycling canconsist of a single musical note, a single repeating rhythmic pattern,or a series of accurate and precise selections from a tempo map. Eachselection in the series of selections can consist of silence, a musicalnote, a plurality of beats, a bar, a plurality of bars, a repeatingrhythmic pattern, or any combination thereof. In some embodiments, avisual portion of the work can provide a point of reference for thecycling such as, for example, during a period of silence in the work, orwhere an audio portion of the work is otherwise insufficient toestablish a point of reference for the cycling, such as where the audioportion is damaged or ineffective.

In some embodiments, the dynamic point reference can function tofacilitate a series of accurate and precise selections from the tempomap, each selection in the series of selections can consist of, forexample, a musical note, a plurality of beats, a bar, a plurality ofbars, a repeating rhythmic pattern, or any combination thereof. Theseries of selections can be learned piecewise by a user to learn anentire work. In some embodiments, the video can include a plurality ofinstructional views of the performing artist performing the work forselection by the user.

In some embodiments, an audio portion of the work can include amulti-track audio portion comprising an isolated audio track consistingof a solo performance that was not obtained through a process ofsubtracting the solo performance from a mixture of performances. In someembodiments, an audio portion of the work can include a multi-trackaudio portion comprising a plurality of isolated audio tracks consistingof a plurality of solo performances that were not obtained through aprocess of subtracting the solo performances from a mixture of theperformances.

In some embodiments, the teachings herein include the development of atempo map, and the tempo map can comprise a mapping of a complex tempo.In some embodiments, for example, the teachings are directed to a methodof creating the instructional work. Such methods can include selectingthe work; and, mapping the audio portion of the work to create a dynamicpoint reference. The dynamic point reference is used to facilitate anaccurate and precise selection, and point-cycling, of a desired portionof the work.

In some embodiments, the teachings are directed to a method for a userto learn a performance through an accurate and precise audiovisualinstructional process. In these embodiments, the method can includeobtaining an audiovisual work having an audio portion, a video portion,and a dynamic point reference for selecting a desired portion of theaudiovisual work. The method can also include selecting the desiredportion of the work using the dynamic point reference and emulating theperformance. The method can also include point-cycling the desiredportion until the desired portion is learned to a satisfaction of theuser.

In some embodiments, the performance is an audiovisual performance of amusical work; wherein, the work has a multi-track audio file derivedfrom a multi-track audio recording. In these embodiments, themulti-track audio file can comprise an isolated instrument audio trackand a residual component track. The isolated instrument audio track cancomprise, for example, a single musical instrument playing a preselectedpiece of music that the user desires to learn on a preselected musicalinstrument. In these embodiments, the emulating can include transformingthe composition of the multi-track audio file to include a gain ratio ofvolumes of (i) the isolated instrument audio track to (ii) the residualcomponent track, wherein the residual component track represents asubtraction of the isolated instrument audio track from the plurality ofaudio tracks. The gain ratio can be selected by the user and, in someembodiments, the gain ratio is a ratio of track volumes that is greaterthan or equal to 0. In addition to the transforming, the emulating caninclude selecting the desired portion, watching the video, listening tothe isolated instrument audio track in the desired portion, and playingthe preselected musical instrument to emulate the desired portion. And,in some embodiments, the emulating further comprises reading a digitalmusical transcription and tablature display, such as an animated motiongraphic or cartoon, of an instrument or performance, to a graphical userinterface. Such data can correspond to the isolated instrument audiotrack. In some embodiments, the custom digital audio file furthercomprises an audible dynamic point reference track. As such, theemulating can also include listening to the audible dynamicpoint-reference track that is designed for the preselected piece ofmusic. In some embodiments, the playback speed can be controlled, or thetempo adjusted, such that the desired portion plays back at a desiredspeed that matches the ability of the user.

In some embodiments, the method further comprises recording the emulatedinstrument audio track on a non-transitory computer readable storagemedium; combining the emulated instrument audio track with the residualcomponent track to create an educational audio file; comparing theuser's emulated instrument audio track to that of the performer'sisolated audio track; and, repeating the emulating, recording,combining, and comparing until the user has learned the preselectedpiece of music on the preselected musical instrument to the user'ssatisfaction.

In some embodiments, the teachings are directed to an audiovisual systemfor learning a performance. The system can comprise, for example, aprocessor; an input device operable to receive audio and video data on anon-transitory computer readable storage medium; and, a databaseoperable to store audiovisual files for access on a non-transitorycomputer readable storage medium. The system can also include an audioengine embodied in a non-transitory computer readable storage medium,wherein the audio engine is operable to transform input audio data tooutput audio data. Likewise, the system can also include a video engineembodied in a non-transitory computer readable storage medium, whereinthe video engine is operable to transform input video data to outputvideo data. Moreover, the system can include a dynamic point referencemodule embodied in a non-transitory computer readable storage medium,wherein the dynamic point reference module is operable to create adynamic point reference for a performance using a tempo map, theperformance having an audiovisual file with the audio data and the videodata. The system can also have an output module embodied in anon-transitory computer readable medium, wherein the output module isoperable to (i) transmit the audio data and the video data to an outputdevice in the form of an accurate and precise selection of a desiredportion of the performance and (ii) transmit a point-cycling of thedesired portion of the performance to a user. The output device isoperable to provide the audio data and the video data to the user,assisting the user in learning a performance.

In some embodiments, the system can include a transformation moduleembodied in a non-transitory computer readable storage medium, whereinthe audio portion comprises a multi-track audio file having an isolatedinstrument audio track and a residual component track, and thetransformation module is operable to transform the audio portion into aratio of (i) the isolated instrument audio track to (ii) the residualcomponent track. In these embodiments, the residual component track canrepresent a subtraction of the isolated instrument audio track from themulti-track audio file, and the transforming can result from a userselecting a gain ratio of volumes between the isolated instrument audiotrack and the residual component track. These embodiments can alsoinclude an emulation recording module embodied in a non-transitorycomputer readable storage medium, wherein the emulation recording moduleis operable to record an emulated audio track. These embodiments canalso include an integration engine embodied in a non-transitory computerreadable storage medium, wherein the integration engine is operable tocombine the emulated instrument audio track with the residual componenttrack to compare the emulated audio track to the isolated instrumentaudio track.

In some embodiments, the input device can include a microphone, and acamera, and the output device can include a speaker and a graphical userinterface. In some embodiments, the output module transmits musictranscription and tablature data to a graphical user interface, themusic transcription and tablature data reflecting variable tempos fromthe multi-track audio file. In some embodiments, the output module has arecalibration function operable to recalibrate an audio data trackoutput to adjust tempo in the output of the audio track data. In someembodiments, the output module has a synchronization function operableto synchronize the music transcription and tablature data display on thegraphical user interface with the isolated instrument audio trackprovided to the listener through the speaker. It should be appreciatedthat the input and output can be an analog or digital audio or video.

In some embodiments, the multi-track audio file further comprises anaudible dynamic point reference track, and the transformation module isoperable to transform the multi-track audio file into a gain ratio of(i) the isolated instrument audio track, (ii) the residual component,and (iii) the audible dynamic point reference track. In someembodiments, the system further comprises a data exchange moduleembodied in a non-transitory computer readable medium, the data exchangemodule operable to exchange data with external computer readable media.

In some embodiments, the system is operable using a hand-held device.And, in some embodiments, the system is operable to function as aparticular machine or apparatus having the additional function oftelecommunications, word processing, or gaming. Moreover, in someembodiments, the system is operable to function as a particular machineor apparatus not having other substantial functions.

In some embodiments, the system consists of a handheld apparatus. And,the handheld apparatus can have no other substantial functions. In someembodiments, the handheld apparatus is a smart phone, or a handheldcomputing device having other substantial functions. In someembodiments, the system consists of a game console apparatus.

BRIEF DESCRIPTION OF THE FIGURES

FIG. 1 shows a general technology platform for the system, according tosome embodiments.

FIGS. 2A and 2B illustrate a processor-memory diagram to describecomponents of the system, according to some embodiments.

FIG. 3 is a concept diagram illustrating the system, according to someembodiments.

FIG. 4 is a diagram of the logic of the system, according to someembodiments.

FIG. 5 illustrates a circuit diagram of the system according to someembodiments.

FIG. 6 illustrates a circuit diagram of the system including a clickfunction, according to some embodiments.

FIGS. 7A and 7B illustrate a musical notation display designed for usewith the system, according to some embodiments.

FIGS. 8A-8C illustrates the system with a variety of functions for avariety of musical instruments, according to some embodiments.

FIG. 9 illustrates a view of a couple dancing in an audio/video dancingdemonstration, according to some embodiments.

FIG. 10 illustrates a display for a graphical user interface offeringselections within a track list of a multi-track audio recording setdesigned for use with the system, according to some embodiments.

FIG. 11 illustrates a display for a graphical user interface offeringinformation on the track list selection in the system, according to someembodiments.

FIG. 12 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which all tracks are selected on a multi-track audiorecording designed for use with the system, according to someembodiments.

FIG. 13 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which all tracks but the click track are selected on amulti-track audio recording designed for use with the system, accordingto some embodiments.

FIG. 14 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which only the isolated instrument audio track and theemulated instrument audio track are selected on a multi-track audiorecording designed for use with the system, according to someembodiments.

FIG. 15 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which only the residual track and the emulated instrumentaudio track are selected on a multi-track audio recording designed foruse with the system, according to some embodiments.

FIG. 16 illustrates a display for a graphical user interface offeringmusical transcription and tablature audio data, in which only thecurrent section of music is shown for a multi-track audio recordingdesigned for use with the system, according to some embodiments.

FIG. 17 illustrates a display for a graphical user interface offeringselection of a section of a piece of music by bar or set of bars in amulti-track audio recording designed for use with the system, accordingto some embodiments.

FIG. 18 illustrates a display for a graphical user interface offering ahelp page for the system, according to some embodiments.

FIG. 19 shows how a network may be used for the system, according tosome embodiments.

DETAILED DESCRIPTION OF THE INVENTION

The systems and methods taught herein are generally directed to adynamic point referencing of an audiovisual performance for an accurateand precise selection and controlled cycling of portions of theperformance. The dynamic point referencing can be used by a learningartist, for example, in analyzing or performing a portion of the workthrough an accurate and precise digital audio/video instructional methodhaving the controlled cycling feature. Such systems and methods will beappreciated, for example, by musicians, dancers, and other enthusiastsof the performing arts.

In some embodiments, the work comprises an audio portion and a videoportion, wherein the video portion includes a performance of the work bya performing artist. In these embodiments, a dynamic point reference canbe included within the musical work for an accurate and preciseselection and point-cycling of a desired portion of the work. Thedynamic point reference can include a tempo map of the musical work thatis adaptable for a plurality of tempos within the musical work. Anexample of tempo mapping can be found, for example, in U.S. Pat. No.7,902,446, which is hereby incorporated herein in its entirety byreference. The terms “artist,” “performing artist,” “performer,”“instructional artist,” “instructor,” “teacher,” and the like, can beused interchangeably in some embodiments.

In some embodiments, the teachings are directed to a method for a userto learn a performance through an accurate and precise audiovisualinstructional process. In these embodiments, the method can includeobtaining an audiovisual work having an audio portion, a video portion,and a dynamic point reference for selecting a desired portion of theaudiovisual work. The method can also include selecting the desiredportion of the work using the dynamic point reference and emulating theperformance. The method can also include point-cycling the desiredportion until the desired portion is learned to a satisfaction of theuser. The term “user” can include any operator of the system which, insome embodiments, can include a “learning artist,” “student,”“observer,” “analyst,” “voter,” “judge,” “teacher”, “instructor,”“competitor”, “competing artist”, “competing performer,” and the like.

In some embodiments, the teachings herein can include the development ofa tempo map, and the tempo map can comprise a mapping of a complextempo. In some embodiments, for example, the teachings are directed to amethod of creating the instructional work of a performing art. Suchmethods can include selecting the work; and, mapping the audio portionof the work to create a dynamic point reference. The dynamic pointreference is used to facilitate an accurate and precise selection, andpoint-cycling, of a desired portion of the work. In some embodiments,the dynamic point reference includes a tempo map of the audio portionthat is adaptable for a plurality of tempos within the work. The term“dynamic point reference” can also be referred to as a “map,” “a tempomap,” “a customized tempo map,” “a timing reference,” “a variable timingreference,” “a manually created and audible, variable timing reference,”“an index,” “indexing,” “a custom index,” “a customized indexing,”“grid,” “tempo/index,” “position point reference,” “variable pointreference,” or the like, in some embodiments.

The terms “audio/video,” “audiovisual,” “audio/visual,” “AV,” and thelike, can be used interchangeably in most embodiments, the termstypically being used to refer to a work having an audio track or audioportion and a corresponding video portion or video portion. The term“corresponding” can refer, for example, to (i) the original source filerecorded with the video; or, (ii) a recreated file in which the audioand/or video has been recreated or added, making the audio/video fileuseful for the teachings provided herein. Audiovideo files can include,for example, a monotrack audio file, a stereo audio file, or amulti-track audio file. A “multitrack file” can include, for example,from about 3 tracks to over 200 tracks, in some embodiments. And, itshould be appreciated that a multi-track audio file can be analog ordigital and, in fact, it can be extracted from a mono- or stereo-mixedfile, or it can be extracted from any type of audio file sharingmultiple instruments. The term “extracted” can be used to refer to aseparation of instruments, in some embodiments, that are combined in anaudio file. One of skill will readily identify such extraction methodsknown in the art including, but not limited to, processes that usephase- or phase-reversal techniques or frequency techniques (e.g.,fast-fourier transform, FFT, discrete fourier transform, DFT, etc),mathematical algorithms, or a combination thereof, for example. Suchextractions can be used with the teachings herein, in some embodiments.

An advantage of the teachings provided herein is the accuracy andprecision available in the selection of a desired portion. Due to thisadvantage, the cycling can include any selected time-frame within thework, such as a period of silence in an audio track, or any one or anyseries of sounds, beats, and bars. In some embodiments, the cycling canconsist of a single musical note, a single repeating rhythmic pattern,or a series of accurate and precise selections from a tempo map. Eachselection in the series of selections can consist of silence, a musicalnote, a plurality of beats, a bar, a plurality of bars, a repeatingrhythmic pattern, or any combination thereof. In some embodiments, avisual portion of the work can provide a point of reference for thecycling such as, for example, during a period of silence in the work, orwhere an audio portion of the work is otherwise insufficient toestablish a point of reference for the cycling, such as where the audioportion is damaged or ineffective. The ability to accurately andprecisely select and cycle a desired portion of a musical work is avaluable aspect of the teachings provided herein. The terms “cycling,”“point cycling,” “rhythmic cycling,” “looping,” and “controlledcycling,” can refer to the repeated playback of a desired portionselected using the teachings provided herein.

The terms “accurate,” “accurately,” and the like, can be used to referto a measurable error, or in some embodiments the absence of ameasurable error, in the difference between a user's actual desiredportion, and the user's ability to select the desired portion from awork using the teachings provided herein. And, the desired portion canbe a portion of an audio track or a related video portion. The terms“precise,” “precisely,” “precision,” and the like can be used to referthe user's ability to reproducibly select the desired portion. The highdegree of accuracy provided through the systems and methods taughtherein produces, in response, likewise results in a high degree ofprecision for the user as well.

In some embodiments, the systems methods provided herein results insubstantially no error in the selecting of the desired portion, suchthat any error may be isolated to fall within a relatively high accuracyof a computing system performing the functions taught herein which, insome embodiments, any error in selecting the desired portion can be lessthan a fraction of a millisecond. Such a computing system error may be,for example, less than about 3 milliseconds in some embodiments, betweenabout 0.5 milliseconds and about 3 milliseconds in other embodiments,and less than perhaps a single millisecond in yet other embodiments. Theterm “desired portion” can be used to refer to a portion of a musicalwork that may range in duration, for example, from about 50 millisecondsto about 20 minutes. In some embodiments, a user can try to select adesired portion having a time duration of play that is very short, forexample, ranging from about 90 ms to perhaps about 0.5 seconds or 1second, making it difficult for the user to select the desired portionin a real-time manner. In this instance, such user error resulting fromuser response time is mitigated by offering the various features taughtherein that include, for example, (i) a “nudge” function that allows theuser to move forward in a preselected number of beats or any timereference, such as a single beat, fraction of a beat, pairs of beats,sets of beats, bars, pairs of bars, sets of bars, and the like; (ii)rules that automatically provide a region around an area selected, suchthat the user can be too slow or too fast in response time and stillcapture the desired portion; and (iii) a tempo adjustment to allow theuser to, for example, slow down a whole work, or particular section tofacilitate ease of selection of the desired portion.

As such, in any event, the accuracy provided by the teachings providedherein are extraordinarily high despite the presence of any potentialuser error, as well as near perfect to perfect (within the bounds of anycomputing system error) in the absence of such user error. One of skillwill appreciate that the high degree of accuracy results in a highdegree of precision that can be realized using the systems and methodstaught herein. Moreover, since the cycling feature is based on the samesystem structure, the cycling feature is likewise highly accurate andprecise. This high level of accuracy and precision allows for anaccurate and precise selection and cycling of a desired of portion of avideo portion in an audio/video work. Accordingly, one of skill willappreciate that the systems and methods provided herein offer a highlyaccurate, highly precise, and high resolution systems and methods thatcan be used in a variety of applications, for example, to analyze andstudy a performance in an audiovisual work.

FIG. 1 shows a general technology platform for the system, according tosome embodiments. The computer system 100 may be a conventional computersystem and includes a computer 105, I/O devices 150, and a displaydevice 155. The computer 105 can include a processor 120, acommunications interface 125, memory 130, display controller 135,non-volatile storage 140, and I/O controller 145. The computer system100 may be coupled to or include the I/O devices 150 and display device155.

The computer 105 interfaces to external systems through thecommunications interface 125, which may include a modem or networkinterface. It will be appreciated that the communications interface 125can be considered to be part of the computer system 100 or a part of thecomputer 105. The communications interface 125 can be an analog modem,isdn modem, cable modem, token ring interface, satellite transmissioninterface (e.g. “direct PC”), or other interfaces for coupling thecomputer system 100 to other computer systems. In a cellular telephone,this interface is typically a radio interface for communication with acellular network and may also include some form of cabled interface foruse with an immediately available personal computer. In a two-way pager,the communications interface 125 is typically a radio interface forcommunication with a data transmission network but may similarly includea cabled or cradled interface as well. In a personal digital assistant,the communications interface 125 typically includes a cradled or cabledinterface and may also include some form of radio interface, such as aBLUETOOTH or 802.11 interface, or a cellular radio interface, forexample.

The processor 120 may be, for example, any suitable processor, such as aconventional microprocessor including, but not limited to, an IntelPentium microprocessor or Motorola power PC microprocessor, a TexasInstruments digital signal processor, or a combination of suchcomponents. The memory 130 is coupled to the processor 120 by a bus. Thememory 130 can be dynamic random access memory (DRAM) and can alsoinclude static ram (SRAM). The bus couples the processor 120 to thememory 130, also to the non-volatile storage 140, to the displaycontroller 135, and to the I/O controller 145.

The I/O devices 150 can include a keyboard, disk drives, printers, ascanner, and other input and output devices, including a mouse or otherpointing device. The display controller 135 may control in theconventional manner a display on the display device 155, which can be,for example, a cathode ray tube (CRT) or liquid crystal display (LCD).The display controller 135 and the I/O controller 145 can be implementedwith conventional well known technology, meaning that they may beintegrated together, for example.

The non-volatile storage 140 is often a FLASH memory or read-onlymemory, or some combination of the two. A magnetic hard disk, an opticaldisk, or another form of storage for large amounts of data may also beused in some embodiments, although the form factors for such devicestypically preclude installation as a permanent component in somedevices. Rather, a mass storage device on another computer is typicallyused in conjunction with the more limited storage of some devices. Someof this data is often written, by a direct memory access process, intomemory 130 during execution of software in the computer 105. One ofskill in the art will immediately recognize that the terms“machine-readable medium” or “computer-readable medium” includes anytype of storage device that is accessible by the processor 120 and alsoencompasses a carrier wave that encodes a data signal. Objects, methods,inline caches, cache states and other object-oriented components may bestored in the non-volatile storage 140, or written into memory 130during execution of, for example, an object-oriented software program.

The computer system 100 is one example of many possible differentarchitectures. For example, personal computers based on an Intelmicroprocessor often have multiple buses, one of which can be an I/O busfor the peripherals and one that directly connects the processor 120 andthe memory 130 (often referred to as a memory bus). The buses areconnected together through bridge components that perform any necessarytranslation due to differing bus protocols.

In addition, the computer system 100 can be controlled by operatingsystem software which includes a file management system, such as a diskoperating system, which is part of the operating system software. Oneexample of an operating system software with its associated filemanagement system software is the family of operating systems known asWindows CEO and Windows® from Microsoft Corporation of Redmond, Wash.,and their associated file management systems. Another example ofoperating system software with its associated file management systemsoftware is the LINUX operating system and its associated filemanagement system. Another example of an operating system software withits associated file management system software is the PALM operatingsystem and its associated file management system. Another example of anoperating system is an ANDROID, or perhaps an iOS, operating system. Thefile management system is typically stored in the non-volatile storage140 and causes the processor 120 to execute the various acts required bythe operating system to input and output data and to store data inmemory, including storing files on the non-volatile storage 140. Otheroperating systems may be provided by makers of devices, and thoseoperating systems typically will have device-specific features which arenot part of similar operating systems on similar devices. Similarly,WinCE®, PALM, IOS or ANDROID operating systems, for example, may beadapted to specific devices for specific device capabilities.

The computer system 100 may be integrated onto a single chip or set ofchips in some embodiments, and can be fitted into a small form factorfor use as a personal device. Thus, it is not uncommon for a processor,bus, onboard memory, and display/I-O controllers to all be integratedonto a single chip. Alternatively, functions may be split into severalchips with point-to-point interconnection, causing the bus to belogically apparent but not physically obvious from inspection of eitherthe actual device or related schematics.

FIGS. 2A and 2B illustrate processor-memory diagrams to describecomponents of the system, according to some embodiments. In FIG. 2A, thesystem 200 shown in FIG. 2 contains a processor 205 and a memory 210(that can include non-volatile memory), wherein the memory 210 includesan audio/video database 215, a transformation module 220, an emulationrecording module 225, an integration engine 230, an output module 235,and an optional video display module 240, which can also be a part ofthe output module 235. The system can further comprise an optional dataexchange module 245 embodied in a non-transitory computer readablemedium, wherein the data exchange module is operable to exchange datawith external computer readable media.

The system includes an input device (not shown) operable to receiveaudio data or video data on a non-transitory computer readable medium.Examples of input devices include a data exchange module operable tointeract with external data formats, voice-recognition software, ahand-held device in communication with the system including, but notlimited to, a microphone, and the like, as well as a camera or othervideo image capture and transmission device. It should be appreciatedthat the input and output can be an analog or digital audio or video.

The audio/video database 215 is operable to store audio or video filesfor access on a non-transitory computer readable storage medium. In someembodiments, the system can store original multi-track audio files,copies of original multi-track audio files, and the like. Any audio orvideo file known to one of skill in the art can be stored including, butnot limited to sound files, text files, image files, and the like. Insome embodiments, the system can access any of a variety of accessibledata through a data exchange module, as discussed above.

Any audio or video format known to one of skill in the art can be used.In some embodiments, the audio file comprises a format that supports oneaudio codec and, in some embodiments, the audio file comprises a formatthat supports multiple codecs. In some embodiments the audio filecomprises an uncompressed audio format such as, for example, WAV, AIFF,and AU. In some embodiments, the audio file format comprises losslesscompression such as, FLAC, Monkey's Audio having file extension APE,WayPack having file extension VW, Shorten, Tom's lossless AudioKompressor (TAK), TTA, ATRAC Advanced Lossless, Apple Lossless, andlossless WINDOWS Media Audio (WMA). In some embodiments, the audio fileformat comprises lossy compression, such as MP3, Vorbis, Musepack,ATRAC, lossy WINDOWS Media Audio (WMA) and AAC.

In some embodiments, the audio format is an uncompressed PCM audioformat, as a “.way” for a WINDOWS computer readable media, or as a“.aiff” as a MAC OS computer readable media. In some embodiments aBroadcast Wave Format (BWF) can be used, allowing metadata to be storedin the file. In some embodiments, the audio format is a lossless audioformat, such as FLAC, WayPack, Monkey's Audio, ALAC/Apple Lossless. Insome embodiments, the lossless audio format provides a compression ratioof about 2:1. In some embodiments, the audio format is a free-and-openformat, such as way, ogg, mpc, flac, aiff, raw, au, or mid, for example.In some embodiments, the audio format is an open file format, such asgsm, dct, vox, aac, mp4/m4a, or mmf. In some embodiments the audioformat is a proprietary format, such as mp3, wma, atrac, ra, ram, dss,msv, dvg, IVS, m4p, iklax, mxp4, and the like.

The transformation module 220 is operable to transform a multi-trackaudio file comprising an isolated instrument audio track and a residualcomponent track into a ratio of (i) the isolated instrument audio trackto (ii) the residual component track, wherein the residual componenttrack represents a subtraction of the isolated instrument audio trackfrom the plurality of audio tracks, and the transforming can resultsfrom a user selecting a gain ratio, for example, between the isolatedinstrument audio track, the residual component track, and a click track.It should be appreciated that a “gain ratio” can be used to refer to auser-controlled variable sound level relationship between the minimum(inaudible) sound volume (infinity:1) to maximum loudness output (0 dBfull scale with a ratio of 1:1). The terms “gain” and “volume” can beused interchangeably in some embodiments, where a gain of “0” can beused, in some embodiments, as a reference for a minimum volume of anaudio portion, track or otherwise; and, a ratio of 0 can be used torefer to a gain in the numerator of 0. For example, a ratio of anisolated instrument audio track to a residual component audio track of 0can mean, for example, that at least the isolated instrument audio trackhas been turned off, or at least down to the minimum volume setting of0. This setting allows the residual component volume, or gain, to beadjusted to an audible level desired by a user. Likewise, a ratio of aresidual component audio track to an isolated instrument audio track of0, for example, can mean that the residual component audio track hasbeen turned off, or at least down to the minimum volume setting of 0,such that the isolated instrument audio track can be adjusted to adesired audible level for play in the absence of the residual componentaudio track.

The “residual component track” can be referred to as “a background audiotrack” or “an emulation audio track,” in some embodiments. The term“metronome track” can be referred to as a “click track,” “manuallycreated, audible, variable timing reference track,” “audible, variabletiming reference track,” “variable timing reference track,” “audible,dynamic point reference track,” “audible, position point referencetrack,” “audible, variable point reference track,” and the like, in someembodiments. It should be appreciated that, in some embodiments, themetronome track can provide a steady tempo. However, the term “metronometrack” can also be used to refer to a track having a variable tempo foruse with the dynamic point referencing taught herein.

The residual track represents a subtraction of the isolated instrumentaudio track from the plurality of audio tracks, and the transforming canresult from a user selecting a between the isolated instrument audiotrack, the residual track, and the click track. The system can alsoinclude an emulation recording module 225 embodied in a non-transitorycomputer readable medium. The emulation recording module 225 is operableto record the user's emulated audio track on a non-transitory computerreadable medium. In some embodiments, the emulation recording module 225can be operable within a single functioning section of a system, such asa single page of a software application. In some embodiments, theemulation recording module 225 can be operable within a plurality offunctioning sections of a system, such as in a plurality of pages of asoftware application, such that the recording can occur quickly at thechoosing of the user without having to move from one section of thesystem to another section of the system.

The system can also include an integration engine 230 embodied in anon-transitory computer readable medium, wherein the integration engine230 is operable to combine the emulated instrument audio track with theresidual track to transform the multi-track audio file into aneducational audio file. In addition, the system can include an outputmodule 235 embodied in a non-transitory computer readable medium,wherein the output module 235 is operable to transmit audio data to anoutput device, which can be a graphical user interface, or videodisplay, which can optionally be supported by a separate video displaymodule 240, or the display can be supported with one or more otheroutput devices by the output module 235. The output device can beoperable to provide audio data to the user, wherein the audio dataassists the user in learning a preselected piece of music.

In some embodiments, the input device comprises a microphone and/orcamera and, in some embodiments, the output module 235 transmits musictranscription and tablature data, or an alternative visualrepresentation that may or may not be traditional transcription andtablature data, such as an animated motion graphic or cartoon, of aninstrument or performance, to a graphical user interface. In someembodiments, the output device comprises a speaker, a graphical userinterface, or both a speaker and a graphical user interface, forexample. And, in some embodiments, the output module has asynchronization function operable to synchronize the music transcriptionand tablature data display on the graphical user interface with theisolated instrument audio track provided to the listener through thespeaker.

The output module 235 can also have a recalibration function operable torecalibrate an audio data track output to correct a latency in theoutput of the audio track data. One of skill will appreciate that timestamps can be used to align, recalibrate, and correct latencies in theoutput of a data stream flow. In some embodiments, the latency iscorrected by time stamping samples of the audio data, where a “sample”is a short wave form of the audio having a length measured by anincrement of time. In some embodiments, a sample is less than a secondlong, (e.g., about 1/100 of a second, or 1/1000 of a second, induration). In some embodiments, the samples can be about 44/1000 of asecond long. An audio track can contain, for example, about 44,000samples per track per second in a high quality sound file. As such, theconcept of a sample and sample resolution is a measure of audioresolution or quality. A lower quality mono audio file can have about22,000 samples per track per second, for example.

Recalibration techniques can be used in some embodiments. Bandwidthlimitations of a computer system, for example, can create variations ortransients in misalignments between tracks and hinder sound quality. Acomputer having a smaller CPU than another computer having a larger CPUbut similar memory capacity can have latency problems that result inperformance problems. In some embodiments, the system can provide anoutput of 4 streaming files, and these files can include (i) a residualcomponent track that does not contain the isolated instrument audiotrack, (ii) an isolated instrument audio track, (iii) an emulatedinstrument audio track, and (iv) a dynamic point reference that canprovide a dynamic point referencing of a variable tempo. In someembodiments, the residual component track, the isolated instrument audiotrack, and the emulated instrument audio track are stereo files and, insome embodiments, the index track is a mono file. Each track in eachfile has it's own timeline, and there can be acceptable variances thatare not noticeable to the ear, but there can also be unacceptablevariances that result in an unacceptable and audible degradation inaudio quality.

Each input file is comprised of an array of samples, and each sample canbe used as a marker in time, since each sample position has an actualposition that will serve as a measure of variance against an idealposition. The recalibration is performed on sets of samples. When agroup of samples is off in time, the system can be designed to correctfor the next set of samples. For example, a system can be designed torecalibrate based on a sample resolution that provides a 0.001 secondaccuracy by measuring the variance of a time stamp on a group of 44samples (0.001 of a second for a 44,000 sample/sec high quality sample)to an ideal time stamp for that sample set. A fast method ofrecalibration was developed to reduce this variance or “latency” in theaudio so that it's not detectable to human being. A good ear, forexample, can hear time variance between tracks of about 1/60 of asecond, and multiple events of variances in a closely related string ofsamples can be cumulative, making it necessary to have a variance atleast an order of magnitude under 1/60 of a second. In some embodiments,the minimum audio resolution in a sample for latency correction shouldbe no less than 300 samples in a segment. In some embodiments, the audioresolution in a sample for latency correction is about 44 samples in asegment. In some embodiments, it was found that the recalibration shouldbe done on each “run through” of the data in the system, where a “runthrough” is an emptying and a filling of a data queue in the system.Between data loads in a queue, the recalibration occurs by measuring thedifference between the actual time of each track and the ideal time ofeach track and a correction is applied between data loads. In someembodiments, the audio queues up several hundred times per second.

The CPU on a handheld computer system can have difficulties concurrentlyprocessing the audio data files described herein. In some embodiments, ahandheld computing system may have latency difficulties whenconcurrently processing more than 2 audio data files. As such, datafiles may require compression. In some embodiments, the data files canbe compressed using a compression technique, for example, such asQUICKTIME by Apple. Other file compression techniques can be used. IMA4can also be used to compress the files in some embodiments. In someembodiments, the system requires at least a 600-700 MHz processor. TheiPhone has a 400 MHz processor, on the other hand, suggesting thatcompressed audio data files may be needed for use of some embodiments ofthe system on the iPhone. The IMA4 compression method compresses theaudio data file to about 25% of file size. An iPAD system can also beused in some embodiments.

In some embodiments, it should be appreciated, however, that the systemcan use pure, uncompressed wave files. Many home PCs, however, may notneed compressed files due to the more powerful processors currentlyavailable for home PCs. The bandwidth of the computer system, i.e. thesize of the CPU and memory will dictate whether compression isnecessary. One of skill in the art will appreciate that certaincompression technologies may be needed in some systems for optimumperformance and that these technologies are readily identifiable andaccessible.

One of skill will appreciate that time stamping of data samples can alsobe used to synchronize between other data streams. In some embodiments,an additional audio data stream is used to provide a digital musicaltranscription and tablature display in the form of a graphical display.This audio data can be synchronized and recalibrated at the same time asthe other audio data.

An audio file, such as a multi-track audio file, can further comprise anindex track. As such, in some embodiments, the transformation module 220can be operable to transform a multi-track audio file into a ratio of(i) the isolated instrument audio track, (ii) the residual track, and(iii) the index track, and a between the isolated instrument audiotrack, the residual track, and the index track can be selected by theuser.

As described above, the system can further comprise an optional dataexchange module 245 embodied in a non-transitory computer readablemedium, wherein the data exchange module is operable to exchange datawith external computer readable media. The data exchange module can, forexample, serve as a messaging module operable to allow users tocommunicate with other users having like subject-profiles, or othersusers in a profile independent manner, merely upon election of the user.The users can email one another, post blogs, or have instant messagingcapability for real-time communications. In some embodiments, the usershave video and audio capability in the communications, wherein thesystem implements data streaming methods known to those of skill in theart. In some embodiments, the system is contained in a hand-held device;operable to function as a particular machine or apparatus having theadditional function of telecommunications, word processing, or gaming;or operable to function as a particular machine or apparatus not havingother substantial functions.

In FIG. 2B, a flowchart shows a variation of the system 200 whereprocessor 205 is in operable connection with the memory 210 and theaudio/video database are in digital media file storage 210,215. Thesystem 200 can also include an audio engine 250 embodied in anon-transitory computer readable storage medium, wherein the audioengine 250 is operable to transform input audio data to output audiodata. Likewise, the system 200 can also include a video engine 255embodied in a non-transitory computer readable storage medium, whereinthe video engine 255 is operable to transform input video data to outputvideo data. Moreover, the system 200 can include a dynamic pointreference module 260 embodied in a non-transitory computer readablestorage medium, wherein the dynamic point reference module 260 isoperable to create a dynamic point reference for a performance using atempo map, the performance having an audiovisual file with the audiodata and the video data. The system 200 can also have an output module(not shown) embodied in a non-transitory computer readable medium,wherein the output module is operable to (i) transmit the audio data andthe video data to an output device in the form of an accurate andprecise selection of a desired portion of the performance and (ii)transmit a point-cycling of the desired portion of the performance to auser. The output device 275 is operable to provide the audio data andthe video data to the user, assisting the user in learning aperformance. The system 200 can also include a notation display engine265 to display music transcription and tablature indexed to the dynamicpoint reference. Moreover, the system 200 can include a user controlinterface 270.

The systems taught herein can be practiced with a variety of systemconfigurations, including personal computers, multiprocessor systems,microprocessor-based or programmable consumer electronics, network PCs,minicomputers, mainframe computers, and the like. The teachings providedherein can also be practiced in distributed computing environments wheretasks are performed by remote processing devices that are linked througha communications network. As such, in some embodiments, the systemfurther comprises an external computer connection through the dataexchange module 245 and a browser program module (not shown). Thebrowser program module (not shown) can be operable to access externaldata as a part of the data exchange module 245.

FIG. 3 is a concept diagram illustrating the system, according to someembodiments.

The system 300 contains components that can be used in a typicalembodiment. In addition to the audio database 215, the transformationmodule 220, the emulation recording module 225, the integration engine230, and the output module 235 shown in FIG. 2, the memory 210 of thedevice 300 also includes a data exchange module 245 and the browserprogram module (not shown) for accessing the external data. The systemincludes a speaker 352, display 353, and a printer 354 connecteddirectly or through I/O device 350, which is connected to I/O backplane340.

The system 300 can be implemented in a stand-alone device, rather than acomputer system or network. In FIG. 3, for example, the I/O device 350connects to the speaker (spkr) 352, display 353, and microphone (mic)354, but could also be coupled to other features. Such a device can havea music state selector 341, an isolated instrument audio track stateselector 342, a residual track state selector 343, a user's emulatedaudio track state selector 344, a learning state selector 345 for theeducational audio track, a bar state selector 346, and a timer stateselector 347 for the index track, with each state selector connecteddirectly to the I/O backplane 340.

In some embodiments, the system further comprises security measures toprotect the subject's privacy, integrity of data, or both. Such securitymeasures are those well-known in the art such as firewalls, software,and the like. In addition, the system can be configured for use in anenvironment that requires administrative procedures and control. Forexample, the system can include an administrative module (not shown)operable to control access, configure the engines, monitor results,perform quality assurance tests, and define audiences for targeting andtrending. Since the system can safely be provided by a network and, insome embodiments, the system is coupled to a network, the securitymeasures can help protect the contents of the system from externalintrusions.

In some embodiments, the system is a web enabled application and canuse, for example, Hypertext Transfer Protocol (HTTP) and HypertextTransfer Protocol over Secure Socket Layer (HTTPS). These protocolsprovide a rich experience for the end user by utilizing web 2.0technologies, such as AJAX, Macromedia Flash, etc. In some embodiments,the system is compatible with Internet Browsers, such as InternetExplorer, Mozilla Firefox, Opera, Safari, etc. In some embodiments, thesystem is compatible with mobile devices having full HTTP/HTTPS support,such as iPhone, PocketPCs, Microsoft Surface, Video Gaming Consoles, andthe like. In some embodiments, the system can be accessed using aWireless Application Protocol (WAP). This protocol will serve the nonHTTP enabled mobile devices, such as Cell Phones, BlackBerries, Droids,etc., and provides a simple interface. Due to protocol limitations, theFlash animations are disabled and replaced with Text/Graphic menus. Insome embodiments, the system can be accessed using a Simple ObjectAccess Protocol (SOAP) and Extensible Markup Language (XML). By exposingthe data via SOAP and XML, the system provides flexibility for thirdparty and customized applications to query and interact with thesystem's core databases. For example, custom applications could bedeveloped to run natively on iPhones, Java or .Net-enabled platforms,etc. One of skill will appreciate that the system is not limited to anyof the platforms discussed above and will be amenable to new platformsas they develop.

In some embodiments, the teachings are directed to a method for a userto learn a performance through an accurate and precise audiovisualinstructional process. In these embodiments, the method can includeobtaining an audiovisual work having an audio portion, a video portion,and a dynamic point reference for selecting a desired portion of theaudiovisual work. The method can also include selecting the desiredportion of the work using the dynamic point reference and emulating theperformance. The method can also include point-cycling the desiredportion until the desired portion is learned to a satisfaction of theuser.

FIG. 4 is a diagram of the logic of an audio portion of the system,according to some embodiments. In some embodiments, the teachings aredirected to a method of learning music through an educationalaudiovisual track, such as an instructional musical work. The methodincludes obtaining 405 a multi-track audio file produced from amulti-track audio recording. The multi-track audio file can be presentedwith a related video portion according to the teachings provided herein.The multi-track audio file can have an isolated instrument audio trackand a residual track, and the isolated instrument audio track can have asingle musical instrument playing a preselected piece of music that auser desires to learn on a preselected musical instrument. The methodincludes transforming 410 the composition of a multi-track audio file toinclude a ratio of (i) the isolated instrument audio track to (ii) aresidual track. The residual component audio track represents asubtraction of the isolated instrument audio track from the plurality ofaudio tracks. The method includes emulating 415 the preselected piece ofmusic by listening to the isolated instrument audio track, watching thecorresponding video portion, and playing the preselected musicalinstrument to create an emulated instrument audio track. Consistent withthe teachings provided herein, the emulating can include the videoinstructional which can be controlled and indexed through the tempo mapas taught herein for use in conjunction with an audio track. Theemulated instrument audio track is recorded 420 on a non-transitorycomputer readable medium and combined 425 with the residual track totransform the custom digital audio file into an educational audio file.The method includes listening 430 to the educational audio track toidentify deficiencies in the emulating by the user. The user repeats 435the emulating, recording, combining, and listening until the user haslearned the preselected piece of music on the preselected musicalinstrument to the user's satisfaction.

In some embodiments, the transforming 410 includes reducing the volumeof the residual track and, in some embodiments, the transforming 410includes reducing the volume of the isolated instrument audio track. Insome embodiments, the video portion is controlled through the tempo mapas taught herein for a user to better understand how to play, andemulate the subtleties in an audio track. The method can furthercomprise selecting one or more bars of the isolated instrument audiotrack to enable the user to focus on emulating a section of thepreselected piece of music. In some embodiments, the method can includethe selection one or more bars to provide a repeated playback of thesection. In some embodiments, the emulating 415 can further comprisereading a digital musical transcription and tablature displaycorresponding to the isolated instrument audio track. And, in someembodiments, the custom digital audio file further comprises an isolatedindex track, and the method further comprises listening to the isolatedindex track that is designed for the preselected piece of music.

The performing artist in an audiovisual work can be amateur orprofessional. For example, the performing artist can be a popularartist, in some embodiments. Likewise, the musical work can be amateuror professional. For example, the musical work can be a popular musicalwork, in some embodiments. And, in some embodiments, the musical workcan be an original, master multi-track recording of a popular,professional artist or group. As such, the performing artist can even bethe popular professional artist or group that performed the original,master multi-track recording.

It should be appreciated that the user can isolate any desired portionof an audio track, and associated video portion, including any one orany series of sounds, beats, and bars. In some embodiments, the desiredportion can consist of a single musical note, a single repeatingrhythmic pattern, or a series of accurate and precise selections fromthe tempo map. Each selection in the series of selections can consist ofa musical note, a plurality of beats, a bar, a plurality of bars, arepeating rhythmic pattern, or any combination thereof.

As described herein, a video display provides a unique way for thelearning artist to more readily learn a piece of music. In someembodiments, the video portion includes a plurality of instructionalviews of the instructional artist performing the musical work forselection by the learning artist.

The musical work can include any format or compilation that one of skillwould find useful as applied to the teachings herein. In someembodiments, the musical work can be a multi-track musical workcomprising an isolated audio track consisting of a solo performance thatwas not obtained through a process of subtracting the solo performancefrom a mixture of performances. Likewise, in some embodiments, themusical work can be a multi-track musical work comprising a plurality ofisolated audio tracks consisting of a plurality of solo performancesthat were not obtained through a process of subtracting the soloperformances from a mixture of the performances.

In some embodiments, an audiovisual file comprises a video portionsynchronized with an audio track, wherein the video portion includes aperformance of a work by a performing artist. In these embodiments, adynamic point referencing method can be included within the musical workfor an accurate and precise selection and cycling of a desired portionof the synchronized audio and video portions by the learning artist. Thedynamic point referencing can include a tempo map of the musical workthat is adaptable for a plurality of tempos within the musical work. Theaccuracy and precision of the selection of the desired portion of thesynchronized audio and video data provides an instructional tool that isvaluable to the learning artist. For example, a dynamic pointreferencing of the musical work can provide accuracy and precision inthe selection of the desired portion when using the methods providedherein. The tempo map can include a plurality of tempos or a complextempo, in some embodiments. It should also be appreciated that the tempomap can represent a fixed tempo in some embodiments, such that the tempomap comprises a single tempo in a substantial portion of a work, andeven consists of a single tempo throughout the work.

The teachings are also directed to a method of creating theinstructional musical work.

The method comprises selecting the musical work, which may containeither synchronized video and audio tracks, or either fileindependently, mapping the musical work to provide a customized tempomap for an accurate and precise selection and of the music by thelearning artist, and indexing the audio track with the tempo map.

The teachings are also directed to a method of learning music through anaccurate and precise digital audio/video instructional process. Themethod comprises obtaining the instructional musical work, the audiotrack or tracks including a multi-track audio file produced from amulti-track audio recording. The multi-track audio file can compriseisolated instrument audio tracks and residual tracks, and the isolatedinstrument audio tracks can comprise multiple single audio tracksplaying a preselected piece of music that a learning artist desires tolearn on a preselected musical instrument. The method also includestransforming the composition of the multi-track audio files to have a of(i) the isolated instrument audio tracks to (ii) a residual track ortracks. In these embodiments, the residual track represents asubtraction of the isolated instrument audio track from the plurality ofaudio tracks, and the gain ratio is selected by the learning artist. Themethod also includes emulating the preselected piece of music byselecting and the desired portion, watching the video portion in thedesired portion, listening to the isolated instrument audio track ortracks in the desired portion, and playing the preselected musicalinstrument to emulate the desired portion to create an emulatedinstrument audio track. Moreover, the method includes recording theemulated instrument audio track on a non-transitory computer readablestorage medium, combining the emulated instrument audio track with theisolated audio track to transform the custom digital audio file into aneducational audio file, listening to the emulated audio track toidentify deficiencies of the emulating by the learning artist, andrepeating the emulating, recording, combining, listening until thelearning artist has learned the preselected piece of music on thepreselected musical instrument to the learning artist's satisfaction. Itshould be appreciated that the can be a ratio of track volumes that isgreater than or equal to 0.

The emulating can include selecting the desired portion, watching thevideo, listening to the isolated instrument audio track in the desiredportion, and playing the preselected musical instrument to emulate thedesired portion. And, in some embodiments, the emulating furthercomprises reading a digital musical transcription and tablature displaycorresponding to the isolated instrument audio track. In someembodiments, the custom digital audio file further comprises an audibledynamic point reference track. As such, the emulating can also includelistening to the audible dynamic point-reference track that is designedfor the preselected piece of music. In some embodiments, the playbackspeed can be controlled, or the tempo adjusted, such that the desiredportion plays back at a desired speed that matches the ability of theuser. Moreover, in some embodiments, the method further comprisesrecording the emulated instrument audio track on a non-transitorycomputer readable storage medium; combining the isolated instrumentaudio track with the residual component track to create an educationalaudio file; comparing the user's emulated instrument audio track to thatof the performer's isolated audio track; and, repeating the emulating,recording, combining, and comparing until the user has learned thepreselected piece of music on the preselected musical instrument to theuser's satisfaction.

One of skill will appreciate that the instructional musical work havingthe audio/video combination provides an excellent learning tool, andthis tool can include other features. In some embodiments, the emulatingfurther comprises reading a digital musical transcription and tablaturedisplay corresponding to the isolated instrument audio track. In someembodiments, the custom digital audio file further comprises an isolatedindex track, the method further comprises listening to the isolatedindex track that is designed for the preselected piece of music. And, insome embodiments, the synchronized audio and video of the desiredportion plays back at a desired speed that matches the ability of thelearning artist.

The teachings can also be directed to an audio/video system for learningmusic. In some embodiments, the system includes a processor, an inputdevice operable to receive audio and video data on a non-transitorycomputer readable storage medium, and a database operable to storesynchronized audio and video instructional files for access on anon-transitory computer readable storage medium. In these embodiments,the system includes a transformation module embodied in a non-transitorycomputer readable storage medium, wherein the transformation module isoperable to transform a multi-track audio file comprising an isolatedinstrument audio track and a residual track into a ratio of (i) theisolated instrument audio track to (ii) the residual track. The residualtrack can represent a subtraction of the isolated instrument audio trackfrom the plurality of audio tracks, and the transforming can result froma learning artist selecting a between the isolated instrument audiotrack, the residual track, and the click track. In these embodiments,the system also includes an emulation recording module embodied in anon-transitory computer readable storage medium, wherein the emulationrecording module is operable to record the learning artist's emulatedaudio track on the computer readable storage medium; and, an integrationengine embodied in a non-transitory computer readable storage medium,wherein the integration engine is operable to combine the isolatedinstrument audio track with the residual track to transform themulti-track audio file into an educational audio file. Moreover, inthese embodiments, the system includes an output module embodied in anon-transitory computer readable medium, wherein the output module isoperable to transmit synchronized audio and video data to an outputdevice in the form of an accurate and precise selection of a desiredportion for a of the desired portion of the synchronized audio and videoportions by the learning artist; and an output device operable toprovide a synchronized audio and video data to the learning artist,wherein the combination and of the synchronized audio and video dataassists the learning artist in learning a preselected piece of music.

It should be appreciated that the input device can comprise an internalor external microphone, an external line or microphone level input, anda camera or other video input device, for example; and, the outputdevice can comprise a speaker, video output device, line output andgraphical user interface. In some embodiments, the output moduletransmits music transcription and tablature data to a graphical userinterface. The output module can also have a recalibration functionoperable to recalibrate an audio data track output to correct for alatency in the output of the audio track data. And, in some embodiments,the output module can have a synchronization function operable tosynchronize the music transcription and tablature data display on thegraphical user interface with the isolated instrument audio trackprovided to the listener through the speaker.

In some embodiments, the audio file further comprises an index track. Inthese embodiments, the transformation module is operable to transformthe multi-track audio file into a of (i) the isolated instrument audiotrack, (ii) the residual track, and (iii) the index track, wherein abetween the isolated instrument audio track, the residual track, and theindex track is selected by the learning artist.

The system can be operable to communicate with external computerreadable media. In some embodiments, the system further comprises a dataexchange module embodied in a non-transitory computer readable medium,wherein the data exchange module is operable to exchange data with anexternal computer readable media.

In some embodiments, the system can be operable using a hand-helddevice.

And, the system is operable to function as a particular machine orapparatus having the additional function of telecommunications, wordprocessing, or gaming. Likewise, the system can be operable to functionas a particular machine or apparatus not having other substantialfunctions.

In some embodiments, the system consists of a handheld apparatus. And,the handheld apparatus can be designed to have no other substantialfunctions. Likewise, the handheld apparatus can be a smart phone, oranother type of handheld computing device having other substantialfunctions.

FIG. 5 illustrates a circuit diagram of the system, according to someembodiments. The solo audio part (the part to be learned) is retained inan audio store 505, and the audio recording of the other parts of thecomposition are retained in an audio store 510. Storage areas 505 and510 may reside in separate devices or in a single storage device, butcan be accessed separately. The progressive retrieval of audioinformation from stores 505 and 510 is controlled by a playbackcontroller 515 such that the various parts of the composition becomesynchronized in time. The solo audio signal can pass through a variablegain element 520 so that it's volume level in an overall output 540 canbe controlled. Similarly, the other audio signals pass through a gainelement 525 and are fed to a mixing element 530, to be combined with thesolo instrument audio signal provided from gain element 520. A similarcircuit diagram can be used for video stores.

An external instrument input element 545 enables an external audiosource, such as the practice instrument, to be included in overalloutput 540. The signal provided by instrument input element 545 passesthrough a gain element 550 before delivery to mixing element 530.

The overall output level can be controlled using a gain element 535,which receives an input signal from mixing element 530 and feeds anoutput signal to overall output 540. The various gain elements can becontrolled directly through user controls or through signals from acontrolling device such as a microprocessor. In some embodiments, otherparts of the musical performance may be stored in separated storageareas to facilitate the learning process or to accommodate multipleplayers.

FIG. 6 illustrates a circuit diagram of the system including a clickfunction, according to some embodiments. An audible timing reference, ordynamic point reference, is included into the signal provided to theoutput of the device. This timing reference may be a periodic sound suchas a click, beep, or a tone, which can be recorded and stored in amethod similar to that used for the other audio signals, or which may besynthesized by an audio signal generator 605 while the musical piece isbeing played. This additional sound is delivered to mixing element 530via a variable gain element 610, which allows the audible level of thedynamic point reference to be adjusted.

In some embodiments, the dynamic point reference can be created from atempo map, such that the dynamic point reference, or indexing, isvariable over time, rather than constant. Many musical works cannot beaccurately mapped using a constant timing reference, as these works canbe found to have, for example, variable and unpredictable tempo changesduring the course of the performance of the musical work. And, this isparticularly true of original musical works, such as those found in theoriginal multi-track master recordings that are the subject of manyembodiments of the teachings provided herein.

Sheet music, for example, can be obtained for such original musicalworks, and these are generally expected to comprise transcriptions of anoriginal and complex variance of tempos into a constant tempo. Theteachings provided herein, however, do not use a constant tempo but,rather, are based on a custom tempo mapping, for example, of an originalrecording providing a dynamic point reference that is substantiallyidentical to the originally recorded work. Traditional sheet music, ordigital sheet music (e.g., pdf scans of sheet music), of popular musicprovide a derivative musical work that is, in most cases, transcribedmuch different than the originally recorded musical work. Traditionalsheet music, for example, generally uses repeated sections that ignorethe nuances that occur in an original musical work. The teachingsprovided herein, however, do not use such repeated sections, but ratherare linear and verbatim representations of the original work transcribedfrom the original, isolated instrument tracks of an original masterproviding a transcription of the work that is transcribed to besubstantially identical to the originally recorded work.

FIGS. 7A and 7B illustrate a musical notation display designed for usewith the system, according to some embodiments. In some embodiments, adisplay provides an animated graphical representation of the musicalnotation relative to the part to be learned, whereby such graphicalrepresentation scrolls along the display area in synchronism with theaudio music signal. In FIG. 7A, for example, a time region 705,710 ofthe display area is marked to identify the present moment, with thegraphical representation of the music moving past time region 705,710 asthe music part is being played. By this arrangement, the display areaenables a player to see musical events in advance that are immediatelyfollowing the present moment. In some embodiments, a recent musicalnotation may also be seen in retrospect. In FIG. 7B, for example, adesired portion of the musical work is accurately and precisely selectedfor cycling, as shown by the broken line, in which the speed of thecycled portion can be adjusted for the skill level of the user.Moreover, the size of the broken line box can be reduced to a very highresolution to capture nearly any desired time increment in the work, forexample, a small set of beats, a single beat, a fraction of beat, andthe like. Accordingly, the apparatus and methods of presenting musicalnotation are advantageous for at least the reason that a user isprovided with a temporal context, tempo speed adjustments, and videomatching of the same for the musical events of interest.

Moreover, and contrary to traditional sheet music notation, thescrolling of the music can be continuous, removing the need for pageturns. And, as described herein, the scrolling of the notation canreflect the variable and unpredictable tempo changes of an original,multi-track master, rather than a derivative work produced using aconstant, or substantially constant, tempo. Having the function ofdigital indexing with the tempo map, as described herein, allows for anaccurate and precise selection of the desired portion of the musicalwork, as well as an accurate and precise cycling at any tempo desired bythe user. A tempo can be referred to as “substantially constant,” forexample, where it was not created to reflect the variable andunpredictable tempo changes of the original, multi-track master. Itshould be appreciated that a transcriptionist of music could vary tempofrom time to time in a musical work, but one of skill would considersuch tempo changes to be substantially constant in view of the variableand unpredictable tempo changes that can occur from the originalperformance by the original artist as found in an original, multi-trackmaster recording.

FIGS. 8A-8C illustrates the system with a variety of functions for avariety of musical instruments, according to some embodiments. In FIG.8A, the system gives the user the standard play 805, rewind 807, record809, and a transport control 810 to control the location and cyclepoints of the video/audio. The transport control 810 also givesmusicians the ability to quickly reference different sections of a song,create custom markers, select point-cycling function 820 that snap to agrid and slow the audio down using a slow down enable button 830 withoutchanging the pitch. The timeline 825 for the musical work can also beindicated, for example, by a horizontal time bar. The point-cycleenablement button 827 allows the user to using the point-cyclingfunction 820. Users can select multiple windows containing various videoangles 840 and rearrange them on the device's screen. Users willappreciate the ability to control the zoom level of each window using azoom control 850 containing the various video angles. It should also benoted that the musician may not know how to read standard notation orguitar tab and could greatly improve his skill using this technique ofidentifying cycle points, choosing the appropriate camera view,adjusting the zoom level and slowing down the track until the identifiedsection has been perfected. At this point, the user can gradually beginto speed up the tempo using a control for adjusting the amount of slowdown 860 and practice the identified part until it can be played atregular speed. FIG. 8B shows that the learning artist can be a drummer,and FIG. 8C shows that the learning artist can be a keyboard player.FIGS. 8A-8C are discussed further in the examples below.

As shown in FIG. 8A, the learning artist can be a guitar player. Aguitar player could quickly and easily select a desired portion of aninstructional musical work, for example, a difficult 2 bar section, andcycle it so that it repeats as many times as desired. A computer screencould display multiple video angles at the same time on different partsof the display. One section of the display, for example, could show aninstructional artist playing, using his fingers with a pick. Anothersection of the display, for example, could show the instructionalartist's fingers on the fret board of the guitar. Other angles could beprovided including, but not limited to, a front view, overhead view, aplayer's point of view, etc.

If the learning artist is still having trouble mastering this selectionhe could also choose to slow down the speed to a point that is withinhis ability to master the section. This would allow the musician toclearly see exactly where he should pick a certain string on one hand orshow which fret to depress with the other hand. He could begin toincrease the tempo as he gains more confidence and eventually play thesection at the original tempo.

As shown in FIG. 8B, a drummer could select a difficult drum-fill in asong as a desired portion of the instructional musical work. Forexample, the methods provided herein allow the drummer to quickly,accurately, and precisely select and cycle, perhaps, a one bar sectionthat he wants to learn. The musical piece can be simple or complex, andthe audio/video compilation would assist in teaching the learning artisthow to master the section.

For example, the video display could show a drummer with an extensivelylarge setup, and the learning artist may want to design a customizedscreen selection that focuses on the toms. And, the design can havesubstantial flexibility, allowing the learning artist to select multipleangles, as well as a variety of magnifications. The learning artist, forexample, may find it appropriate to zoom in on the over head video angleto best understand the selection of toms that an instructional artist isstriking in the selected section. As shown in FIG. 8B, the drums can beindicated by numbers 811, and the sequence of play can be explained inthis manner, for example, to the user.

As shown in FIG. 8C, a keyboard player may find it useful to examine theexact fingering that an instructional artist is using in a desiredportion of a performance. Visualizing an exact fingering can be veryuseful to a learning artist, as the selection of which fingers to use onthe selected keys can make it much easier to perform a difficult passagein a musical work.

Having the ability to select the desired portion with accuracy andprecision, as well as view a synchronized video portion corresponding tothat desired portion, and being able to reproducibly select and repeat aaround the desired portion, can provide an unprecedented valuablelearning tool. Such information will be appreciated by one of skill as avaluable learning aid, as well as by the learning artist or, in fact, byany keyboard player that didn't learn to play using a teacher orutilizing the classical fingering techniques.

As shown in FIG. 8A, the system 800 can include an instruction for anyinstrument, such as a guitar 802, and performance can be rated 872. Thesystem 800 can include a video camera feature 861 that allows users tovideo themselves playing and superimpose or compare the user capturedvideo to instructional video, and this feature 861 can also generateanimations of the user, in some embodiments, using a motion-capturecamera. Viewing angle adjustment 804, and a panning feature 806, can bea standard feature. In some embodiments, the system includes a narrativevideo/audio track through a narrative camera function 862 that gives theuser information or instruction pertaining to the song. In someembodiments, the system includes a narrative teacher feature 863 toselect from a variety of different narrative teachers that may includecelebrities or popular artists. Moreover, the system may also have anaccuracy feature 864 that automatically compares the audio/video from auser's performance to the instructional audio/video as a measure of theuser's performance. The accuracy feature 864 can even serve as real timefeature that notifies a user when a wrong note has been performed.

In some embodiments, the system includes an option that stops the userwhen he makes a mistake. In some embodiments, a tolerance option 865 candefine the amount of tolerance that system will allow before stoppingthe user or indicating that a mistake has been made. And, in someembodiments, a multi-channel playback engine would let the user isolateand balance the audio level of the instructional instrument, theresidual component track, the click track and any other instruments thatmay be isolated within the session.

In some embodiments, the system includes a click track 866 that helpsthe user keep time with the song. And, in some embodiments, the system800 has an ability to provide a visual representation 867 of thesuggested finger placement on instrument (i.e. #1 corresponds to thumb,#2 index figures, etc.). Moreover, the system can have an a notationenabling function 868 to provide the user with a notation screen as wellas a video screen as an additional learning tool.

In some embodiments, the system has a note to note function 869, withthe capability for users to advance note to note (set to a specific notevalue—⅛, 16^(th), quarter, half notes, etc.) at their own pace using astep advance mode. This would allow the learning of difficult musicalpassages by viewing the displays and hearing the notes in a non-realtime “move-on when you're ready” method.

The system could also include a plurality of communities or socialnetworks that include like-minded musicians or fans that can comparetheir skill level or exchange tips and lessons. As shown in FIG. 8A, aplayer may submit and share through a community share function 871,sharing a recorded performance 872 with the community and display thegrading or skill level that has been achieved on a per song basis or anoverall average ability rating. In some embodiments, the performance maybe graded by the host or celebrity narrative chosen.

A user could have a video chat enabling feature 873 as an access to theonline community to get direct feedback or tips from other usersutilizing a video chat system. And, in some embodiments, a player mayreceive points for his performance and achieve a level of status in thecommunity. The system could also be designed such that a player mayexchange his points with other members or redeem his points for avariety of goods or services within the community or the company. Insome embodiments, a player may be rewarded or graded on his appearanceas it pertains to certain communities or predetermined genres.

In some embodiments, the system could include an instrument soundmodeling option 874 that could synchronize a sound modeling processor orfx processor that matches the tone of the chosen song and automaticallyadjusts and changes with the unique tonality of each particular sectionof a song. And, in some embodiments, the system could also give theusers feedback on the quality or their sound as it pertains to theirchosen instrument or the selected song.

The systems taught herein can be in a device that can be configured foruse with any musical instrument, for example, guitar, piano, drums, or avocals. For example, such a device can be configured for use with apiano. The device can be substantially limited to the teachings hereinand include a housing or container of any shape, such as a shapedesigned to sit on top of a piano in place of standard sheet music. Amusic display may be included as a liquid crystal display (LCD) screenor other type of display screen, and one or more transport buttons suchas, for example, a play button, a stop button, and a pause button, canbe provided. Such a device can comprise a number of controls, which maybe configured as knobs or other similar state selectors known in theart. The device can include a first group of state selectors that relateto the “blender” function of the device and are structured to controlthe user's input, the music without the piano contribution, and thepiano itself. A second group of state selectors can be provided torelate to the “master” function of the device and control volume, clicktrack, and tempo. A third group of state selectors can be provided tocontrol the on/off function of the device and may include indicatorlights, light intensity control, and additional playback controls, forexample. The device can also include one or more speakers, a soundmodule for electric pianos, and one or more ports for connecting thedevice to other units such as, for example, USB ports, phono jacks, andpower jacks, or perhaps musical instruments, such as electric organs andguitars, for example. In some embodiments, a USB port may be used toconnect the device to a computer system. In some embodiments, forexample, the USB port allows for downloading of audio data to a largercomputer memory storage location. In some embodiments, data may also besupplied to the device and and/or stored in removable data memory cards.

In some embodiments, the multi-track audio files are produced fromoriginal multi-track recordings, and these recordings may originate onanalog tape, such as analog multi-track tape (e.g. 1 track to 24tracks), a digital tape format (e.g. pulse code modulation, PCM, digitaltape format). In some embodiments, an analog tape format is firsttransformed into a digital recording and a multi-track audio file isproduced from the digital recording. In some embodiments, the originalmix is recreated by taking all of the different tracks and making themix to simulate the original recording. The mixing can be a manualprocess and can be done with an analog console, new digital console, orthe mix can be done on a computer using basically any mixing techniqueknown to one of skill. In some embodiments, older analog tapes need tobe restored, such as by a baking procedure, before attempting arecreation.

It should be appreciated that the teachings can apply to any piece ofmusic containing virtually any musical instrument including, but notlimited to string instruments, brass instruments, woodwind instruments,percussion instruments, and vocals. In some embodiments, pieces of musichaving variable tempos, rhythms, and beats can be learned with more easedue to the manually created and variable metronome function, as well asthe manually created audio files having superior isolated instrumentaudio track quality. In some embodiments, songs are complicated and havechanging or otherwise un-isolatable beats that would be more difficultto learn without the teachings provided herein.

The user will often have a set of multi-track audio files to learn onthe system.

FIG. 10 illustrates a display for a graphical user interface offeringselections within a track list of a multi-track audio recording setdesigned for use with the system, according to some embodiments. TrackList Display 1000 shows the selections in a multi-track audio recordingset or track list 1050. The user chooses an audio file and proceeds tothe menu bar 1005 to select a function from home page 1010, volume/faderpage 1015, music transcription and tablature 1020, cycling page 1025,and help page 1030.

The user can access an information page regarding the piece of musicselected from the track list, such as the band, song title, album,tempo, and tuning. FIG. 11 illustrates a display for a graphical userinterface offering information on the track list selection in thesystem, according to some embodiments. Information Display 1100 providesthe user with the selection bibliographic information 1105 containinginformation on the band, song title, and album, and selection technicalinformation 1110 provides information on the tempo and tuning for theselection. Play function 1150 allows the user to begin learning theselection.

The volume/fader page 1015 has several functions that enables a user toeffectively learn and mix music. FIGS. 12-15 show variousfunctionalities of the volume/fader page 1015. FIG. 12 illustrates adisplay for a graphical user interface offering state selections fortrack faders, volume fader control, and transport selections, in whichall tracks are selected on a multi-track audio recording designed foruse with the system, according to some embodiments.

Volume/fader display 1200 provides the functionality of track on/off(i.e. mute) control 1205 for each track file. The functionality of thevolume/fader controls 1210 is provided by the volume indicators 1215 toindicate sound pressure level and the faders 1220,1225,1230,1235 toadjust volume in the manner of a potentiometer or digitometer, forexample. Transport section 1250 provides a time bar 1255 to indicate aposition in the piece of music and can also contain markings, such ascolors, to indicate the intro, pre-chorus, verse, solo, bridge, outro,chorus, and middle section, for example.

The transport section 1250 also provides several state selectionfunctions: a rewind 1260, pause 1265, fast forward 1270, stop 1275 asnormal transport control state settings; cycle 1280 to allow a user torepeat a desired section of the piece of music; and slow 1285 to allowthe user to slow the song by a predetermined amount and serve the useras a function commonly known as a speed control, tempo adjustment, ortempo control. In some embodiments, the slow 1285 function can be adefault setting (e.g. to slow by some percentage, such as 50%, 75%, orthe like) and, in some embodiments, the user can define a desired speedsetting. As such, in some embodiments, the playback controls can be tothose found on about any tape deck or video cassette recorder, such as“previous”, “next”, “play”, “pause”, and “record”. And, in someembodiments the playback controls include a “cycling” function to allowthe user to cycle a particular section or measure over and over againuntil the user is satisfied in the learning experience.

In some embodiments, the transport section 1250 can be operable within asingle functioning section of a system, such as a single page of asoftware application. In some embodiments, the transport section 1250can be operable within a plurality of functioning sections of a system,such as in a plurality of pages of a software application, such that thetransporting can occur quickly at the choosing of the user withouthaving to move from one section of the system to another section of thesystem. A music label and timer 1290 is also provided as a referencedata point for the user.

In some embodiments, the mixing of audio can be handled on an individualsample per track basis. Each track can be represented individually,having to keep track of its own samples, duration, levels and peaks,gain, and time. Once each track can be initialized and loaded out of itsfile, and then handed off to a subsystem to decode the compression andmake samples. An example of a subsystem can include, for example Apple'sCOREAUDIO subsystem. After the samples are made available, the track canthen be assigned to a master track handler object referred to as themixer object, and saved recorded audio sessions can be loaded at thistime as well. The mixer object primes the subsystem and initializes theoutput. In these embodiments, the touching of the “play” button can beused to start the mixer in its function of combining the audio withinbuffers, where the mixer calls out to each track asking it for its nextframe of audio. The audio frames can be added to the playback buffer andenqueued, and all timing on the audio can then be synced to allow theaudio to be synced to a subnote level to reduce or eliminate creepwithin tracks.

In some embodiments, the audio recording can be handled by a mannersimilar to the individual audio playback, where extensions to record toa file and store its place within a song can be used. Once the userclicks a record function, a recorder object can be built, and the objectcan then start a file and initialize the recording. Once initialized,the recording class can store the current play time within the song tothe measure and begin. The user can tell the device to stop recording,and the object can then mark that time and store the duration of thedata stream in a settings file. The audio data is then flushed to thefile, a track handler object can then be created with its start time andduration set, and the mixer can be updated to allow future playback ofthe recorded audio along with the rest of the prerecorded audio.

FIG. 13 illustrates a display for a graphical user interface offeringstate selections for track faders, volume fader control, and transportselections, in which all tracks but the click track are selected on amulti-track audio recording designed for use with the system, accordingto some embodiments. FIG. 14 illustrates a display for a graphical userinterface offering state selections for track faders, volume fadercontrol, and transport selections, in which only the isolated instrumentaudio track and the emulated instrument audio track might be selected ona multi-track audio recording designed for use with the system,according to some embodiments. FIG. 15 illustrates a display for agraphical user interface offering state selections for track faders,volume fader control, and transport selections, in which only theresidual track and the emulated instrument audio track are selected on amulti-track audio recording designed for use with the system, accordingto some embodiments.

A user can benefit by reading music while playing. FIG. 16 illustrates adisplay for a graphical user interface offering musical transcriptionand tablature audio data, in which only the current section of music isshown for a multi-track audio recording designed for use with thesystem, according to some embodiments. Music transcription and tablaturepage 1600 provides the current transcription and tablature 1605 in abright display and the upcoming transcription and tablature 1610 in adim display, where the current represents the music currently playing,and the upcoming represents the music immediately following the musiccurrently playing. Each measure, for example, can have a timecode andduration that represents where in the song's timeline that measure isplayed. Using that information along with the song's current playbackposition, the tablature can be synchronized along with any playingaudio. Each measure can be designed to display notes or chords and whichstrings or frets to use. In some embodiments, a user may desire astandard music scale for display. The musical transcription andtablature can also be a dynamic, scrolling display, in some embodiments.

A user can also benefit from a function that allows for an isolation ofa particular section of a piece of music in an audiovisual work, as wellas a cycling of that section to enable the user to focus and practice onthat section of the music. FIG. 17 illustrates a display for a graphicaluser interface offering selection of a section of a piece of music bybar or set of bars in a multi-track audio recording designed for usewith the system, according to some embodiments. Cycling page 1700displays section 1705 of the digital audio file, and bar 1710 or bars1715, for example, can be isolated and selected for playback by theuser. The user can then use the methods taught herein to focus and learnthe particular section 1705, bar 1710, or bars 1715, for example. Assuch, in some embodiments, the playback can also be controlled through a“reel screen”, where every measure is segmented. In such embodiments,any audio the user has recorded can also be displayed on the screen inthe measures in which it exists to allow the user to quickly find thataudio and listen to their play of that session as graphed over theoriginal audio data. In some embodiments, a “tape” can be graphicallydisplayed to show the markings of the playback tracking bar. And,consistent with the teachings herein, a video display can complement thesystem, in some embodiments.

The user may have questions, and as such, a help page is always useful.FIG. 18 illustrates a display for a graphical user interface offering ahelp page for the system, according to some embodiments. Help page 1800is a simple depiction of information that a user can obtain from thesystem.

FIG. 19 shows how a network may be used for the system, according tosome embodiments. FIG. 19 shows several computer systems coupledtogether through a network 1905, such as the internet, along with acellular network and related cellular devices. The term “internet” asused herein refers to a network of networks which uses certainprotocols, such as the TCP/IP protocol, and possibly other protocolssuch as the hypertext transfer protocol (HTTP) for hypertext markuplanguage (HTML) documents that make up the world wide web (web). Thephysical connections of the internet and the protocols and communicationprocedures of the internet are well known to those of skill in the art.

Access to the internet 1905 is typically provided by internet serviceproviders (ISP), such as the ISPs 1910 and 1915. Users on clientsystems, such as client computer systems 1930, 1950, and 1960 obtainaccess to the internet through the internet service providers, such asISPs 1910 and 1915. Access to the internet allows users of the clientcomputer systems to exchange information, receive and send e-mails, andview documents, such as documents which have been prepared in the HTMLformat. These documents are often provided by web servers, such as webserver 1920 which is considered to be “on” the internet. Often these webservers are provided by the ISPs, such as ISP 1910, although a computersystem can be set up and connected to the internet without that systemalso being an ISP.

The web server 1920 is typically at least one computer system whichoperates as a server computer system and is configured to operate withthe protocols of the world wide web and is coupled to the internet.Optionally, the web server 1920 can be part of an ISP which providesaccess to the internet for client systems. The web server 1920 is showncoupled to the server computer system 1925 which itself is coupled toweb content 1995, which can be considered a form of a media database.While two computer systems 1920 and 1925 are shown in FIG. 19, the webserver system 1920 and the server computer system 1925 can be onecomputer system having different software components providing the webserver functionality and the server functionality provided by the servercomputer system 1925 which will be described further below.

Cellular network interface 1943 provides an interface between a cellularnetwork and corresponding cellular devices 1944, 1946 and 1948 on oneside, and network 1905 on the other side. Thus cellular devices 1944,1946 and 1948, which may be personal devices including cellulartelephones, two-way pagers, personal digital assistants or other similardevices, may connect with network 1905 and exchange information such asemail, content, or HTTP-formatted data, for example. Cellular networkinterface 1943 is coupled to computer 1940, which communicates withnetwork 1905 through modem interface 1945. Computer 1940 may be apersonal computer, server computer or the like, and serves as a gateway.Thus, computer 1940 may be similar to client computers 1950 and 1960 orto gateway computer 1975, for example. Software or content may then beuploaded or downloaded through the connection provided by interface1943, computer 1940 and modem 1945.

Client computer systems 1930, 1950, and 1960 can each, with theappropriate web browsing software, view HTML pages provided by the webserver 1920. The ISP 1910 provides internet connectivity to the clientcomputer system 1930 through the modem interface 1935 which can beconsidered part of the client computer system 1930. The client computersystem can be a personal computer system, a network computer, a web TVsystem, or other such computer system.

Similarly, the ISP 1915 provides internet connectivity for clientsystems 1950 and 1960, although as shown in FIG. 19, the connections arenot the same as for more directly connected computer systems. Clientcomputer systems 1950 and 1960 are part of a LAN coupled through agateway computer 1975. While FIG. 19 shows the interfaces 1935 and 1945as generically as a “modem,” each of these interfaces can be an analogmodem, isdn modem, cable modem, satellite transmission interface (e.g.“direct PC”), or other interfaces for coupling a computer system toother computer systems.

Client computer systems 1950 and 1960 are coupled to a LAN 1970 throughnetwork interfaces 1955 and 1965, which can be ethernet network or othernetwork interfaces. The LAN 1970 is also coupled to a gateway computersystem 1975 which can provide firewall and other internet relatedservices for the local area network. This gateway computer system 1975is coupled to the ISP 1915 to provide internet connectivity to theclient computer systems 1950 and 1960. The gateway computer system 1975can be a conventional server computer system. Also, the web serversystem 1920 can be a conventional server computer system.

Alternatively, a server computer system 1980 can be directly coupled tothe LAN 1970 through a network interface 1985 to provide files 1990 andother services to the clients 1950, 1960, without the need to connect tothe internet through the gateway system 1975.

Through the use of such a network, for example, the system can alsoprovide an element of social networking, whereby users can contact otherusers having similar subject-profiles. In some embodiments, the systemcan include a messaging module operable to deliver notifications viaemail, SMS, and other mediums. In some embodiments, the system isaccessible through a portable, single unit device and, in someembodiments, the input device, the graphical user interface, or both, isprovided through a portable, single unit device. In some embodiments,the portable, single unit device is a hand-held device. In someembodiments, the systems and methods can operate from the server to auser, from the user to a server, from a user to a user, from a user to aplurality of users, in an MMO environment, from a user to a server to auser, from a server to a user (or plurality of users) and a teacher (orplurality of teachers), or a server to a plurality of users and aconductor, for example. The teachers or conductors can be eitherreal-time teachers or conductors, simulated teachers or conductors, orrecorded teachers or conductors, in some embodiments. The network canprovide text and/or audio for real-time messaging, posting of messages,posting of instructional, posting of news or other items of a relatedinterest to the users, and the like.

It should also be appreciated that a network can include traditionalnetwork media. For example, television cable, optical fiber, satellite,and the like, in which such digital information can be transmitted tousers. Such use of traditional media will facilitate use of theteachings provided herein with traditional media entertainment channels,such as reality TV, amateur talent shows and competitions, and the like.

The following examples are illustrative of the uses of the presentteachings. It should be appreciated that the examples are for purposesof illustration and are not to be construed as otherwise limiting to theteachings.

Example 1. Custom Mapping for the Instructional Musical Work

One of skill in the art of music production will understand that mappingis a skill obtained through experience and is not easy. There's nothingobvious about mapping, and there's no existing system that offers thecustom mapping approach to instructional musical works, methods, andsystems as taught herein. And, complex mapping can be used to provide acontrol for the transport of the learning artist to the desired portionsof the musical works taught herein.

The easiest example of mapping can include the use of a transient orbeat, for example, with drums, from the start to the end of a musicalwork. The skilled artisan could identify a first beat and map it withsubsequent beats, such as first kick, then a first snare, a second kick,and then a second snare, representing one bar in quarter notes. That's asimple tempo map.

The tempo map is based on “distance” that is measured in time betweenbars, or sometimes between beats, e.g., in milliseconds. The distancebetween bar 1 and bar 2 could be, for example. 119.5 beats per minute,between bar 2 and bar 3 could then be 119.38 beats per minute, where theskilled artisan can capture subtle fluctuations in tempo. The capturingof such fluctuations, whether subtle or not, allows for a mapping thatallows a learning artist to locate exactly where the beat is, e.g. adownbeat or an upbeat.

What if there are no drums? For this purpose, the skilled artisan coulduse another instrument, such as a guitar, in search of a downbeat andupbeat, for example. Many musical transients may provide a means foridentifying quarter notes, eighth notes, etc, using any point ofreference known to one of skill, and the choice depends on thecomplexity of the work or the tempo.

What if no tempo at all for some time, such as 8 bars? In this case, theskilled artisan could take the last beat of a section and the first beatof a following section and measure the distance between the selectedbeats.

What if song starts with an ad-lib vocal with no time reference at all?In this case, the skilled artisan could look at the overall tempo of asong, and then use expert interpretation to select a tempo, followed byan iterative process that includes, for example, manually shuffling,adjusting, and laying-out the map.

Tempo mapping functions that are available through PROTOOLS or othersoftware programs do not provide preconfigured custom tempo mapping setsthat will easily and automatically align to any given song as taughtherein within. As will be appreciated by one of skill, as well as thelearning artist, a predefined custom tempo map can provide a quick andeasy solution that provides accuracy and precision not otherwiseavailable.

Example 2. The Use of the Audio Tempo Mapping to Index a CorrespondingVideo Portion.

This example illustrates how to implement the teachings herein to anaudiovisual musical work.

Overdubbing to a Master Recording

This example shows how to use the methods provided herein combining theaudio track derived from a classic multi-track master recording, and theaudio/video performance of an instructor. Our example uses the originalmulti-track audio from Jimi Hendrix' “Purple Haze” and an audio/videoperformance of modern guitar legend Steve Vai demonstrating Jimi'stechnique.

The original multi-track master tapes were transferred to high qualitymulti-track audio files and mixed to sound indistinguishable from theoriginal version of the song. The original guitar performance and theaccompanying band track are both exported separately as stereo audiofiles so that the user will have discrete volume level control overthese elements. Using these isolated audio components, a dynamic pointreference is constructed as described herein. By utilizing theindividual instrument tracks from this original multi-track recording,the dynamic point reference is created using a variety of notes, beats,and transients.

Steve Vai enters the audio/video studio where his rendition of theclassic Jimi

Hendrix song will be recorded with multiple camera angles and highquality audio recording equipment. In addition to the band track (withoriginal guitar removed, for example, using the teachings describedherein and in U.S. Pat. No. 7,902,446, which is hereby incorporatedherein in its entirety by reference), Mr. Vai will use the dynamic pointreference as an accurate audio guide to perform his guitar in sync withthe original band track. Either the audio/video recording of Steve orthe audio of the Hendrix song may benefit from very subtle editing tomatch the 2 audio tracks in certain sections that will be very pleasingto the user.

Taking the finished and edited audio/video files of the performance andauthoring it in sync with the original Hendrix tracks, the user canchoose the desired camera angle for video playback and adjust the audioplayback levels of the band audio track, Steve Vai's performance audiotrack, the original Hendrix audio track and the metronome click track(directly derived from the dynamic point reference). A user can nowselect a desired portion of the video to emulate, and the user'sselection will automatically move to the nearest cycle point and begincycling through the selection. The cycling, for example, can be furtherdefined by a default cycle point selection which might be 2 beats, 1bar, 2 bars or an entire section such as a verse, chorus, or the entiresong. The dynamic point reference provides the point cycle feature withthe information it needs so the user can quickly select a desiredsection of the audio/video and watch the same performance over and overagain, and can do so with the synchronized audio cycling. In this way,the user can very quickly learn every technique that Steve Vaidemonstrated on the Hendrix song. The user can select a small 1 barsection with the point cycle feature turned on in advance and as thesection is perfected, the user can select the next closest start pointor end point allowing the user the ability to change his start and endselection quickly and proficiently to the next closest cycle point atthe beginning or ending of the current selection. The user will benefitfrom the ability to slow down the speed of the audio/video performanceas desired without changing the pitch of the audio and examine thedetail and nuances of Steve Vai performing his rendition of Purple Haze.As such, this allows the user to emulate this performance in slow motionwith a great degree of detail and confidence and, once mastered, theuser may now be able to emulate the performance at regular speed.

Example 3. Virtuoso Performance

This example shows how to use the methods provided herein with extremelycomplex songs by a virtuoso performer learning to play the mostchallenging compositions. Typically, songs of this nature lendthemselves to a more detail-oriented learning experience as compared tomore simple forms of music. The dynamic point reference as describedherein and the use of multiple camera angles showcasing hand position,posture, instrument handling, and other pertinent performance detailscan be observed once a dynamic point reference as described herein isapplied to any format of audio files using point cycling and extremelyslow speed adjustments until the difficult section is mastered andperfected. The user could plug his instrument into the user-definedrecordable track where he will emulate the virtuosic performance.

Example 4. Variable Point Reference Over a Steady Click

This example shows how to use the methods provided herein can be usedwith a constant tempo that never changes. A dynamic point reference asdescribed herein can be derived from an audio/video performance thatoriginated against a constant tempo generated from a drum machine orcomputer music program. It may be desirable to add a new section at anew tempo before during or after the existing audio/video with aconsistent rhythm. It may be desirable to construct an introduction foran audio/video performance where the tempo starts very slow (50 beatsper minute) and gradually speeds up to the constant tempo which could be120 beats per minute. This same concept could be applied and attached tothe end of an audio/video performance where the tempo slowing changesfrom the constant tempo of 120 beats per minute back to the original 50beats per minute at the start. The dynamic point reference makes itpossible to cycle the introduction section that is slowly speeding up,the main section that is at a constant tempo, or the newly constructedending that is gradually slowing down. The user could also plug hisinstrument into the useable recordable track where he will record hisown performance.

Example 5. Classical Performance With A String Quartet.

This example shows how to use the methods provided herein with anaccomplished classical string quartet. There are many physicaltechniques that are specifically relevant to classical musicians whoplay instruments like violin, viola, cello, contra bass etc. A stringplayer's vibrato, bowing, positioning, hand placement and many otherfactors that happen very quickly by the accomplished string player arevery difficult to comprehend due to the speed and complexity of manydifficult musical pieces. However, these nuances and articulations caneasily be observed through an audio/video system that allows a learningmusician to point cycle the desired sections of a song utilizing thedynamic point reference as described herein. Once the cycle point isselected at the desired section, the proper camera angle is selected andthe speed is adjusted slow enough to see and hear the desired section,it is possible to emulate the performer very accurately while practicingthe selection over and over until the user can adjust the speed back toits original state and play it with confidence and dexterity. This is avery useful feature that can accelerate the user's ability as theperformer can motivate the user when a teacher is not available orduring practice sessions in between one-on-one lessons with a teacher.Any of the four instruments of the string quartet can benefit from thiscycling method and slow speed option.

Example 6. Teacher/Student Musical Instruction

This example shows how to use the methods provided herein with a musicalinstructor or teacher and a user who is the student. Hiring a teacherwho specializes in one-on-one student instruction can be expensive andmany students cannot afford more than 1 or 2 lessons per week. Teacherswill commute to the students house or the student will travel to theteachers house or studio, or in some cases a school. Personal one on oneinstruction allows a teacher to give immediate feedback to the student.The teacher may give the student assignments and songs to practice andperfected until the next lesson and many times the student will practicebad habits and make the same mistakes over and over again until the nextlesson when the teacher points out the mistakes.

The dynamic point reference as described herein and the use of multiplecamera angles showcasing hand position, posture, instrument handling,and other pertinent performance details, combined with multi-track audiofiles with isolated instrument tracks offers the student a solution forthis problem. Using the dynamic point reference as described herein, thestudent can methodically use the point cycling for each bar of a songfrom start to finish using any of the camera angles and listening to theaudio at regular speed or slowing the tempo to assist in the learningprocess. The student has a video and audio reference to help reduce thechances of practicing the same mistakes over and over again untilcorrected by the teacher at the next in person lesson.

The teacher can recommend content for the student or the teacher canproduce his own content and be the performer in the audio/video contentthat is used by the student. A portable tablet device for example anApple ipad or Android based device is a great platform for the methodsprovided herein. Using any of the popular analytics tools currentlyavailable combined with the methods provided herein, a teacher couldtrack a student's progress in detail during the times in betweenlessons. For example, a teacher could get reports remotely through anetwork or internet that give details of how often the student practicesin general, which songs are being practiced and the length of time ofpractice per song.

Specific details can also be examined as a diagnostic, for example,identifying the sections of the song that the student is practicingusing the point cycling and slowing down the tempo. This is valuableinformation that a teacher can use to recommend certain exercises fordexterity, fingering or other exercises to help the student obtain theskills required for the problematic sections. The student can also usethe recording function to make an audio/video compilation, or just theaudio portion or video portion separately to post on a server ordirectly email to the teacher. The teacher can choose to listen to thestudent's performance streaming from a server, or import the filelocally to the teachers tablet or computer, and like the student, canuse the point cycling and slow the tempo to make very accurateobservations about the student's performance. The audio will assist theteacher for grading the correct notes being played or, in the case of asinger, the teacher can analyze the pitch and other nuances. The videowill assist the teacher to help determine many performance-relatedissues. For example, if the student is playing piano, determiningwhether the fingering is correct. Likewise, if the instrument is aviolin, the teacher could observe the vibrato or the position of thefingering. Vocal students could be observed for breathing and the shapeof the mouth during certain sections. Many of these examples could alsobe applied to dance, where the teacher could not only monitor how often,and what sections, the student is practicing in addition to therecording and uploading or emailing of performance data for the teacherto evaluate.

Example 7. The Learning Artist Is A Member Of A Band Or OrchestraPerforming Together “Virtually”.

One of skill will appreciate that the systems and methods taught hereincan be provided in a streaming audio/video setting in which members of aband or orchestra can play together in a virtual setting, in which theconductor can be a streaming video portion that is indexed, consistentwith the teachings provided herein, to an audio track. As with the otherembodiments taught herein, the audio track used for the indexing can beany audio track and, in some embodiments, a percussion audio track canbe used. As such, each virtual member of the band or orchestra can belocated in an independent remote setting, for example, such as theirhome or other local venue, and the members can use an internet settingto perform together. The combined performances can be recorded andanalyzed by the band or orchestra in a manner similar to that of a groupsetting with the exception that the recording can also offer an accurateand precise selection of a desired portion of the recorded work, alsohaving the ability to cycle the portion for analysis and study of thework at a high resolution.

Regardless of the information presented, the system exemplifies thebroader concept of an accurate and precise digital audio/videoinstructional system. The system can provide a powerful and uniquelearning experience and, in some embodiments, it can process multimediain the form of text, images, video, and sound. In some embodiments, theuser can customize the system, such as choosing interfaces, colors,language, music preferences and categories, etc. The user is allowed toenter preferences into the system in order to customize visual displaysthat present the information the user in a personalized way. In someembodiments, the system includes a multimedia interaction of one or moreof text and video; sound and diagrams, pictures, or images; sound; andvideo. In some embodiments, the system and it's database can include anyof a variety of system libraries that contain organized sets of any of avariety of information of value to users. Moreover, can information canbe obtained from external data sources, whereby plug-ins and APIs can bedesigned to allow integration with third party systems and exchange datawith external data sources. The external data sources can be used toprovide information on demand, to update existing information stored inthe system libraries, or both. Some portions of the detailed descriptionare presented in terms of operations of the system. The operations arethose requiring physical manipulations of physical quantities resultingin a useful product being produced. In other words, a transformation isoccurring, in some embodiments. In some embodiments, the transformationcan be particular to the use of a particular machine or apparatusdesigned for that transformation. Usually, though not necessarily, thesequantities take the form of electrical or magnetic signals capable ofbeing stored, transferred, combined, compared, and otherwisemanipulated. It has proven convenient at times, principally for reasonsof common usage, to refer to these signals as bits, values, elements,symbols, characters, terms, numbers, or the like. All of these andsimilar terms are to be associated with the appropriate physicalquantities and are merely convenient labels applied to these quantities.Unless specifically stated otherwise, discussions utilizing terms suchas “processing” or “computing” or “calculating” or “determining” or“displaying” or the like, refer to the action and processes of acomputer system, or similar electronic computing device, thatmanipulates and transforms data represented as physical (electronic)quantities within the computer system's registers and memories intoother data similarly represented as physical quantities within thecomputer system memories or registers or other such information storage,transmission or display devices.

Moreover, the teachings relate to a system for performing the operationsherein. This system may be specially constructed as an apparatusdesigned solely for the required purposes, or it may comprise a generalpurpose computer selectively activated or reconfigured by a computerprogram stored in the computer. Such a computer program may be stored ina non-transitory computer readable storage medium, such as, but is notlimited to, any type of disk including floppy disks, optical disks,CD-ROMs, and magnetic-optical disks, read-only memories (ROMs), randomaccess memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, orany type of media suitable for storing electronic instructions, and eachcoupled to a computer system bus. It should also be appreciated that, insome embodiments, the methods and displays presented herein are notnecessarily inherently related to any particular computer or otherapparatus. Various general purpose systems may be used with programs inaccordance with the teachings herein, or it may prove convenient toconstruct more specialized apparatus to perform the methods of someembodiments. The required structure for a variety of these systems willbe apparent to one of skill given the teachings herein. The lawcurrently limits claims to those that do not claim signals, and so theteachings are directed to non-transitory computer readable media as percurrent practice. However, should the law change and rule that claimscan include signals, some embodiments can also include a systemcomprising signals, such as wireless signals of RF devices, for example.In addition, the techniques are not described with reference to anyparticular programming language, and various embodiments may thus beimplemented using a variety of programming languages. Accordingly, theterms and examples provided above are illustrative only and not intendedto be limiting; and, the term “embodiment,” as used herein, means anembodiment that serves to illustrate by way of example and notlimitation.

Example 8. The Learning Artist Is A Dancer.

This example illustrates how the learning artist can be a dancer. Likethe performance of music, the performance of dance can also includesubtleties in movements that create a unique, pleasing, or otherwisedesirable expression. As a musician may have difficulty identifying andmastering a subtlety in the playing of a desired portion of a musicalwork, the dancer may have a similar difficulty identifying and masteringa subtlety found in dancing with the desired portion of the musicalwork. Using the teachings provided herein, the dancer can accurately andprecisely select the desired portion of the musical work that correlateswith a likewise accurate and precise video display of the dancing movesassociated with the desired portion that the dancer wants to learn. Thedancer can cycle that portion at a range of tempos that match thedancer's skill level, repeating the movements until learned to thedancer's satisfaction. This tool can be used by the beginner or expertdancer in learning all or any portion of a dance.

FIG. 9 illustrates a view of a couple dancing in an audio/video dancingdemonstration, according to some embodiments. Many adults enjoy ballroomdancing as a form of entertainment, and some take it quite seriously. Astate-of-the-art method of learning ballroom dancing, for example,typically involves either purchasing a diagram of dance steps that youcan follow-by-number, attending a dancing class in a local forum, orsome combination of these methods. Currently, there is no existingmethod by which dancers can obtain an indexed version of a musical workthat can be accurately and precisely parsed, and adjusted for a desiredtempo; the musical work also having a likewise parseable, indexed videodemonstration of the dance steps that they are trying learn.

Classic AudioNideos Performance

This example shows how to use the methods provided herein using avintage audio/video performance where the audio is a single track monoor 2 track stereo source that was recorded on the analog audio channelof the video recorder or recorded on an external device synchronized tothe video recording unit. This example, in particular, uses aperformance of Elvis Presley performing on a television show in the late1950's where the mono audio source is used to construct the dynamicpoint reference as described herein.

The audio/video file is transferred from its original analog source to acomputer and the audio file is used to construct the dynamic pointreference by using a variety of notes, beats and transients combinedwith visual cues from the video; for example: body movements providing adynamic reference point if the audio is damaged or non-existent. Onceconstructed, the dynamic point reference knows where the desired beatsare located, and a count-off can be constructed using an average cyclepoint calculation from the first few bars with a defined rhythm.

The user can utilize the point cycling feature while the audio/video isplaying and if the default setting is 1 bar, the point cycling willautomatically begin and snap to the nearest bar and repeat the 1 barselection. The point cycling automatically adjusts to the nearest cyclepoint and cycles through a pre-defined cycle length that can be a usersetting preference in the setup menu. A user can manually choose anylength of cycling, for example, an entire section such as a verse,chorus, or the entire song. Once the user defines the cycle length itcan be viewed over and over again with the synchronized audio, and slowdown the tempo making it easier for the user to learn every move ornuance just the way Elvis performed it that night on television. Theuser can easily change the start and end cycle point and move to thenext closest cycle point at the beginning or end of the currentselection.

The user will appreciate the ability to slow down the speed of theaudio/video performance as desired without changing the pitch of theaudio and examine the detail and nuances of Elvis on stage performinghis classic moves, allowing the user to emulate this famous performancein slow motion with a great degree of detail and confidence. Onceperfected, the user can change back to the normal speed and repeat thisprocedure until the performance can be emulated to the user'ssatisfaction.

Modern Solo Freestyle

This example shows how to use the methods provided herein forprofessional and aspiring dancers for modern electronic beat-infusedmusic that has unexpected tempos, start and stops and recorded livethrough a multi-channel audio recorder and an independent and separatevideo recorder. In this example, the audio and video are recorded onseparate devices and synchronized to an external clock source such asSMPTE time code developed by the Society of Motion Picture andTelevision Engineers. The video and audio are later synchronized in themixing process using a synchronization device and time code such asSMPTE.

The multi-track audio source is used to construct the dynamic pointreference as described herein. The audio content on the many differenttracks that make up the multi-track recording offer many different beatsand timing references to construct the dynamic point reference. Avariety of notes, beats and transients are used in the construction ofthe dynamic point reference and can be combined with visual cues fromthe video, if helpful, for example, in a case where the audio is damagedor lacks an obvious dynamic point reference. Once constructed, thedynamic point reference detects where the desired beats are located anda count-off can be constructed using an average cycle point calculationfrom the first few bars of a defined dynamic point reference andinserted before the start point of the song. This inserted piece can beoverlapped with the first bar to help ensure a good dynamic pointreference.

A user can employ the point cycling feature while the video is playingand select the cycle points that are desirable to emulate. The pointcycling automatically adjusts to the nearest cycle point and cyclesthrough a pre-defined cycle length that can be a user setting preferencein the setup menu. Complicated rhythmical dance techniques are easilyrepeated using point cycling at slower speeds and user-selectableintervals to absorb challenging routines. These dance sequences can beviewed at various camera angles allowing the user to see the performancefrom different perspectives that can be desirable as many dancers usemirrors at different angels while rehearsing new material. The pointcycling feature allows a user to find the desired angle while the trackis cycling the selection over and over.

Couples Dance

This example shows how to use the methods provided herein fortraditional dancing that has become very popular with TV shows likeDancing with the Stars where couples dance competitively. The dynamicpoint reference as described herein would be greatly beneficial forsomeone who would like to learn Tango, Salsa, Ballroom or any one of thepopular dance styles. Point cycling of any portion of popular dancewould be invaluable to an aspiring dancer to learn, practice and controlthe viewing options. Point Cycling and the ability to slow the tempo ofany section will help to master the techniques of any dance style. Userscan view the exact body movements at the optimum point of view that ismost conducive to learning.

The user can employ the point cycling feature to select favoritemovements or problematic and hard-to-learn sections using the speedcontrol feature that allows the user to slow the selected section down.The audio not only retains the same pitch, the video is always in syncwith the music which is the pulse that drives the dancer. This methodwill accelerate the learning process for both dancers but will alsoallow each dancer to work independently when they are not together forpractice.

Choreographed Dance Groups

This example shows how to use the methods provided herein using achoreographed dance group like a Broadway musical or a modern danceensemble. A top choreographer is hired to produce a series complex danceroutines that are set to music that has many fluctuations in tempo whichhe will demonstrate while being recorded with a high quality audio/videorecorder and a music play back system generating sound from a pair ofspeakers in the room so he can hear the music and is also recorded bythe video/audio recorder. He performs each complex dance routine and theaudio/video performance is transferred to a computer and the audio fileis used to construct the dynamic point reference as described herein.

Once completed, the finished content will be authored to a portabletablet computer for use during production of a 25 piece dance ensemble.The leader of the dance ensemble gives each of the 25 dancers a tabletwith the complex audio/video dance piece and places them on a stand infront of the dancers. Each of the 25 tablets are set to external syncmode and controlled by the leader or the choreographer who controls eachor the 25 devices from his master device by a wireless network.

The leader or the choreographer is now controlling each dancer's tablet,selecting cycle points of any section of the complex dance routine and,where necessary, slowing down the tempo during the difficult sectionsallowing each dancer to watch and practice together with the entire 25piece ensemble. Each dancer can also take his/her tablet home topractice the dance routine using the point cycle feature to practice thedifficult sections of the dance piece, record a digital video ofthemselves to compare to the performer that they are emulating. Thisexample shows how the methods provided herein can dramatically save timefor a large group of dancers to learn a complex dance piece for aBroadway show, Las Vegas review or any performance by a group ofdancers.

Avant Garde Routine

This example shows how to use the methods provided herein for anabstract audio/video dance routine which may be rehearsed and performedwith ethereal music soundtracks that lack the strong beat definitionthat exists in other forms of music. In cases like these, dance movesneed to be mapped based on the performer's motion as opposed to themusic itself. The variable point reference is derived from the rhythm orselected visual cues of the dancer yet still in sync with theaccompanying music. User-selected segments of the performance canutilize the point cycling and also be slowed down to intuitively learnthe exact movements in relation to the internal rhythm of the performingdancer.

Example 9. The learning artist is a contestant in a performing artscontest, and the systems taught herein can be used by the general publicin rating the performers.

The use of the system can be applied in several forums. Some members ofthe general public, for example, may have an interest in observing amusical performance or dance performance. The media has identified astrong interest in the public with regard to rating such performances.Examples of such forums include, but are not limited to, American Idol,Dancing with the Stars, and America's Got Talent, to name only a few.New versions of such television shows appear to continue development bythe major networks. The systems, methods, and datafiles provided hereincan be used by the networks that produce the shows and present them tothe general public, for example, to offer the public with a way tofurther enjoy and rate performances. A user could download a particularperformance or set of performances and compare them for purposes ofanalyzing the performances and voting for their favorite performance.

We claim:
 1. An audiovisual work including a video keyboard performance,the work comprising: a video portion and an audio portion, wherein thework includes a video keyboard performance of a musician displayingcombinations of finger movements that follow the audio portion; and, adynamic point reference grid within the work for an accurate and preciseselection and point-cycling of a desired portion of the work by a user,wherein the dynamic point reference includes a custom tempo map of theaudio portion that is adaptable for a plurality of tempos within thework; wherein the video portion and the audio portion are stored on anon-transitory computer readable medium with instructions for executionby a processor; the audio portion comprises a recording of a musicalinstrument having a fluctuation in tempos; the custom tempo map isconfigured to include a mapping of the distance between bars or beats,the distance measured as time; where, the mapping captures thefluctuation in tempos and composes the dynamic point reference grid forthe accurate and precise selection of a point of reference forpoint-cycling of the desired portion of the work.
 2. The work of claim1, wherein the audio data comprises an originally recorded work.
 3. Thework of claim 1, wherein the audio data comprises a prerecorded audiodata.
 4. The work of claim 1, wherein the desired portion comprises asingle musical note indexed to the custom tempo map.
 5. The work ofclaim 1, wherein the desired portion consists of a single repeatingrhythmic pattern indexed to the custom tempo map.
 6. The work of claim1, wherein the dynamic point reference grid is configured to facilitatea series of accurate and precise selections from the custom tempo map,each selection in the series of selections consisting of a musical note,a plurality of beats, a bar, a plurality of bars, a repeating rhythmicpattern, or any combination thereof.
 7. The work of claim 1, wherein thevideo includes a plurality of instructional views of the musicianperforming the work for selection by the user.
 8. The work of claim 1,wherein the work includes a multi-track audio portion comprising anisolated audio track consisting of a solo performance.
 9. The work ofclaim 1, wherein the work includes a multi-track audio portioncomprising a plurality of isolated audio tracks consisting of aplurality of solo performances.
 10. The work of claim 1, wherein thecustom tempo map comprises an assignment of a transient to a period ofsilence in the audio track having the fluctuation in tempos.
 11. Amethod of creating the work of claim 1, the method comprising: selectingthe work with the fluctuation of tempos; and, custom-mapping the audioportion of the work to create the dynamic point reference to facilitatean accurate and precise selection, and point-cycling, of the desiredportion of the work.
 12. A method of analyzing or emulating a keyboardperformance through an accurate and precise audiovisual process, themethod comprising: obtaining an audiovisual work having an audio portionhaving a fluctuation in tempos, a video portion having a video keyboardperformance of a musician displaying combinations of finger movementsthat follow the audio portion, and a dynamic point reference grid havinga custom tempo map of the audio portion for selecting a desired portionof the audiovisual work; selecting a point of reference in the desiredportion of the work, the point of reference defined by the dynamic pointreference grid; analyzing or emulating the performance; and,point-cycling the desired portion; wherein, the custom tempo map isconfigured to include a mapping of the distance between bars or beats,the distance measured as time; where, the mapping captures thefluctuation in tempos and composes the dynamic point reference grid foran accurate and precise selection of a point of reference for thepoint-cycling of the desired portion of the work.
 13. The method ofclaim 12; wherein, the audiovisual work has a multi-track audio filederived from a multi-track audio recording; the multi-track audio filecomprises an isolated instrument audio track and a residual componenttrack, the isolated instrument audio track comprising a single musicalinstrument playing a preselected piece of music that the user desires tohear when analyzing or emulating the keyboard performance; the analyzingor emulating includes transforming the composition of the multi-trackaudio file to include selecting a gain ratio of volumes of (i) theisolated instrument audio track to (ii) the residual component track,wherein the residual component track represents a subtraction of theisolated instrument audio track from the plurality of audio tracks; and,selecting the desired portion, watching the video, listening to theisolated instrument audio track in the desired portion, and analyzing oremulating the desired portion.
 14. The method of claim 13, the methodfurther comprising: recording an emulated keyboard video track on anon-transitory computer readable storage medium; combining the user'skeyboard video track with the residual component track to create aneducational audio file; and, comparing the emulated keyboard video trackto that of the performer's keyboard video track.
 15. The method of claim12, wherein the method further comprises emulating the audio portionwhich is controlled and indexed through the custom tempo map, whereinthe keyboard performance includes an instrument performance of the audioportion.
 16. The method of claim 12, wherein the desired keyboard videoportion includes a musical note, a plurality of beats, a bar, aplurality of bars, a repeating rhythmic pattern, or any combinationthereof, of the isolated instrument audio track.
 17. The method of claim12, wherein the emulating further comprises reading a digital musicaltranscription and tablature display corresponding to the isolatedinstrument audio track.
 18. The method of claim 12, wherein the audioportion further comprises an audible dynamic point reference track, themethod comprising listening to the audible dynamic point-reference trackthat is designed for the preselected piece of music.
 19. The method ofclaim 12, wherein the desired portion plays back at a desired speed thatmatches the ability of the user.
 20. An audiovisual system for analyzingor emulating a keyboard video track displaying combinations of fingermovements that follow an audio portion, the system comprising: aprocessor; an input device operable to receive audio and video data on anon-transitory computer readable storage medium; a database operable tostore audiovisual files for access on a non-transitory computer readablestorage medium; an audio engine embodied in a non-transitory computerreadable storage medium, wherein the audio engine is operable totransform input audio data to output audio data; a video engine embodiedin a non-transitory computer readable storage medium, wherein the videoengine is (i) operable to transform input video data to output videodata having a keyboard video track displaying combinations of movementsthat follow the audio portion and (ii) configured to display an indexedversion of the keyboard video track that can be accurately and preciselyparsed, and adjusted for a desired tempo; a dynamic point referencemodule embodied in a non-transitory computer readable storage medium,wherein the dynamic point reference module is operable through a customtempo map to create a dynamic point reference grid for the videokeyboard performance having a video portion and an audio portion, thedynamic point reference grid indexed to the custom tempo map of theaudio portion, the audio portion comprising a recording of a musicalinstrument having a fluctuation in tempos; an output module embodied ina non-transitory computer readable medium, wherein the output module isoperable to (i) transmit the audio data and the video data to an outputdevice in the form of an accurate and precise selection of a desiredportion of the keyboard video track and (ii) transmit a point-cycling ofthe desired portion of the keyboard video track to a user; and, theoutput device operable to provide the audio data and the video data tothe user; wherein, the custom tempo map is configured to include amapping of the distance between bars or beats, the distance measured astime; where, the mapping captures the fluctuation in tempos and composesthe dynamic point reference grid.
 21. The system of claim 20, whereinthe dynamic point reference is constructed to include a transient in thecustom tempo map representing a point of silence as a reference pointfor the point-cycling of the desired portion.
 22. The system of claim20, further comprising a transformation module embodied in anon-transitory computer readable storage medium, wherein the audioportion comprises a multi-track file having an isolated instrument audiotrack and a residual component track, and the transformation module isoperable to transform the audio portion into a ratio of (i) the isolatedinstrument audio track to (ii) the residual component track, theresidual component track representing a subtraction of the isolatedinstrument audio track from the multi-track file, and the transformingresulting from a user selecting a gain ratio of volumes between theisolated instrument audio track and the residual component track; anemulation recording module embodied in a non-transitory computerreadable storage medium, wherein the emulation recording module isoperable to record an emulated keyboard video track; an integrationengine embodied in a non-transitory computer readable storage medium,wherein the integration engine is operable to combine the emulatedkeyboard video track with the audio track.
 23. The system of claim 20,wherein the input device comprises a microphone and a camera, and theoutput device comprises a speaker and a graphical user interface. 24.The system of claim 20, wherein the output module transmits musictranscription and tablature data to a graphical user interface, themusic transcription and tablature data reflecting the fluctuation intempos from the multi-track audio file.
 25. The system of claim 20,wherein the output module has a recalibration function operable torecalibrate an audio data track output to adjust tempo in the output ofthe audio track data.
 26. The system of claim 20, wherein the outputmodule has a synchronization function operable to synchronize the musictranscription and tablature data display on the graphical user interfacewith the isolated instrument audio track provided to the listenerthrough the speaker.
 27. The system of claim 22, wherein the multi-trackaudio file further comprises an audible dynamic point reference track,and the transformation module is operable to transform the multi-trackaudio file into a gain ratio of (i) the isolated instrument audio track,(ii) the residual component, and (iii) the audible dynamic pointreference track.
 28. The system of claim 20 further comprising a dataexchange module embodied in a non-transitory computer readable medium,wherein the data exchange module is operable to exchange data withexternal computer readable media.
 29. The system of claim 20, whereinthe system is operable using a hand-held device.
 30. The system of claim20, wherein the system is configured to display a set of emulatedkeyboard performances for the user to compare them for purposes ofanalyzing the performances.
 31. The system of claim 20, wherein thesystem includes a custom tempo map to reflect the variable andunpredictable tempo changes of an original, multi-track masterrecording.
 32. The system of claim 20, wherein the system consists of ahandheld apparatus.
 33. The system of claim 20, wherein the system hasan external choreographer control configured to allow a choreographer to(i) select a section of the performer's keyboard video track to pointcycle or (ii) select a tempo for a musician among a group of musiciansto follow in the process of learning a keyboard piece.
 34. The system ofclaim 28, wherein the system is configured to include a voting or ratingfunction for the user in the analyzing of the keyboard performance andtransmitting the voting or rating using the data exchange module to theexternal computer readable media.
 35. The system of claim 20, whereinthe system consists of a game console apparatus.